{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* By: Harkishan Singh Baniya\n",
    "* Email: harkishansinghbaniya@gmail.com\n",
    "* Reference (Paper): https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3517595\n",
    "* Reference (Book) : Machine Learning for Asset Managers by Dr. Marcos Lopez De Prado"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustered Feature Importance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of these notebook is demostrate the Clustered Feature Imporatance, a feature importance method suggested by **Dr. Marcos Lopez de Prado** in the [paper](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3517595) and the book Machine Learning for Asset Managers. The aim of CFI is to cluster similar features and apply the feature importance analysis at the cluster level. This way clusters are mutually dissimilar and the method is tends tame the substitution effect and  by using information theory along we can also reduce the multicollinearity of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Support us on Patreon: https://www.patreon.com/HudsonThames\n",
      "\n",
      "MlFinLab needs you! We need your help for us to keep on maintaining and implementing academic research based on \n",
      "financial machine learning (for open-source). In order for us to continue we need to raise $4000 of monthly donations\n",
      "via Patreon - by December 2020. If we can't reach our goal, we will need to adopt more of a paid for service. We thought\n",
      "that the best and least impactful course of action (should we not reach our goal) is to leave the package as open-source\n",
      "but to make the documentation (ReadTheDocs) a paid for service. This is the ultimate litmus test, if the package is a \n",
      "value add, then we need the community to help us keep it going.\n",
      "\n",
      "Our road map for 2020 is to implement the text book: Machine Learning for Asset Managers by Marcos Lopez de Prado, \n",
      "as well as a few papers from the Journal of Financial Data Science. We are hiring a full time developer for 3 months \n",
      "to help us reach our goals. The money that you, our sponsors, contribute will go directly to paying salaries and other \n",
      "expenses such as journal subscriptions and data. \n",
      "\n",
      "We need your help to continue maintaining and developing this community. Thank you for using our package and we \n",
      "invite you to join our slack channel using the following link:\n",
      "https://join.slack.com/t/mlfinlab/shared_invite/zt-c62u9gpz-VFc13j6da~UVg3DkV7~RjQ\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# General Imports \n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import mlfinlab as ml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Clustered Feature Importance or CFI algorithm can be implemented in a two step process as mentioned in the book.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step - 1 : Features Clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As first step we need to generate the clusters or subsets of features we want to analyse with feature importance methods. This can be done using feature cluster module of mlfinlab. It uses various parameters to generating feature clusters as in the book."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The algorithm projects the observed features into a metric space by applying a dependence matric function either correlation based or information theory based. Information-theoretic metrics have the advantage of recognizing redundant features that are the result of nonlinear combinations of informative features (i.e. multicollinearity). \n",
    "* Next, we need to determine the optimal number of clusters. The user can either specify the number cluster to use, this will apply a hierarchical clustering on the defined distance matrix of dependence matrix for a given linkage method for clustering, or the user can use the ONC algorithm which uses K-Means clustering, to automate the task of either getting the optimal number of clusters or get both optimal number of clusters and cluster compositions. \n",
    "\n",
    "But the *caveat* of these process is that some silhouette scores may be low due one feature being a combination of multiple features across clusters. This is a problem, because ONC cannot assign one feature to multiple clusters. Hence, the following transformation may help reduce the multicollinearity of the system:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlfinlab.util.generate_dataset import get_classification_data \n",
    "from mlfinlab.clustering.feature_clusters import get_feature_clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generating a synthetic dataset for testing \n",
    "# We generate 40 features, 5 informative  ('I_') , 30 redudent ('R_') and rest (5) noisy ('N_') features with 10000 rows of samples \n",
    "# Redundent features are those which share large amount of information among each other and also with informative features \n",
    "# That is the redudent features are those with substitution effect.\n",
    "X, y = get_classification_data(n_features=40,n_informative=5,n_redundant=30,n_samples=10000,sigmaStd=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>I_0</th>\n",
       "      <th>I_1</th>\n",
       "      <th>I_2</th>\n",
       "      <th>I_3</th>\n",
       "      <th>I_4</th>\n",
       "      <th>N_0</th>\n",
       "      <th>N_1</th>\n",
       "      <th>N_2</th>\n",
       "      <th>N_3</th>\n",
       "      <th>N_4</th>\n",
       "      <th>...</th>\n",
       "      <th>R_20</th>\n",
       "      <th>R_21</th>\n",
       "      <th>R_22</th>\n",
       "      <th>R_23</th>\n",
       "      <th>R_24</th>\n",
       "      <th>R_25</th>\n",
       "      <th>R_26</th>\n",
       "      <th>R_27</th>\n",
       "      <th>R_28</th>\n",
       "      <th>R_29</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2.105359</td>\n",
       "      <td>2.861661</td>\n",
       "      <td>0.104159</td>\n",
       "      <td>0.686149</td>\n",
       "      <td>1.369429</td>\n",
       "      <td>-0.868903</td>\n",
       "      <td>-1.297125</td>\n",
       "      <td>-0.160205</td>\n",
       "      <td>-0.481024</td>\n",
       "      <td>0.841338</td>\n",
       "      <td>...</td>\n",
       "      <td>0.777001</td>\n",
       "      <td>2.207398</td>\n",
       "      <td>0.655435</td>\n",
       "      <td>2.142781</td>\n",
       "      <td>0.004543</td>\n",
       "      <td>0.815639</td>\n",
       "      <td>2.109815</td>\n",
       "      <td>2.762477</td>\n",
       "      <td>0.863380</td>\n",
       "      <td>0.588109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>-0.330754</td>\n",
       "      <td>1.464379</td>\n",
       "      <td>-1.405119</td>\n",
       "      <td>0.396713</td>\n",
       "      <td>-1.722305</td>\n",
       "      <td>0.471952</td>\n",
       "      <td>-1.443687</td>\n",
       "      <td>-0.433773</td>\n",
       "      <td>0.123114</td>\n",
       "      <td>-0.102970</td>\n",
       "      <td>...</td>\n",
       "      <td>0.355247</td>\n",
       "      <td>-0.295893</td>\n",
       "      <td>0.445317</td>\n",
       "      <td>-0.376633</td>\n",
       "      <td>-1.364581</td>\n",
       "      <td>0.393410</td>\n",
       "      <td>-0.647636</td>\n",
       "      <td>1.561900</td>\n",
       "      <td>0.517651</td>\n",
       "      <td>0.460849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>-0.461334</td>\n",
       "      <td>-0.160432</td>\n",
       "      <td>-2.169501</td>\n",
       "      <td>-0.137535</td>\n",
       "      <td>0.398229</td>\n",
       "      <td>-0.278979</td>\n",
       "      <td>-1.860566</td>\n",
       "      <td>0.909540</td>\n",
       "      <td>-0.396742</td>\n",
       "      <td>2.455228</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.172824</td>\n",
       "      <td>-0.421676</td>\n",
       "      <td>-0.132882</td>\n",
       "      <td>-0.351309</td>\n",
       "      <td>-2.156682</td>\n",
       "      <td>-0.214241</td>\n",
       "      <td>-0.399131</td>\n",
       "      <td>-0.081916</td>\n",
       "      <td>-0.092813</td>\n",
       "      <td>-0.166722</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 40 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        I_0       I_1       I_2       I_3       I_4       N_0       N_1  \\\n",
       "0  2.105359  2.861661  0.104159  0.686149  1.369429 -0.868903 -1.297125   \n",
       "1 -0.330754  1.464379 -1.405119  0.396713 -1.722305  0.471952 -1.443687   \n",
       "2 -0.461334 -0.160432 -2.169501 -0.137535  0.398229 -0.278979 -1.860566   \n",
       "\n",
       "        N_2       N_3       N_4  ...      R_20      R_21      R_22      R_23  \\\n",
       "0 -0.160205 -0.481024  0.841338  ...  0.777001  2.207398  0.655435  2.142781   \n",
       "1 -0.433773  0.123114 -0.102970  ...  0.355247 -0.295893  0.445317 -0.376633   \n",
       "2  0.909540 -0.396742  2.455228  ... -0.172824 -0.421676 -0.132882 -0.351309   \n",
       "\n",
       "       R_24      R_25      R_26      R_27      R_28      R_29  \n",
       "0  0.004543  0.815639  2.109815  2.762477  0.863380  0.588109  \n",
       "1 -1.364581  0.393410 -0.647636  1.561900  0.517651  0.460849  \n",
       "2 -2.156682 -0.214241 -0.399131 -0.081916 -0.092813 -0.166722  \n",
       "\n",
       "[3 rows x 40 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There is no feature/s found with low silhouette score. All features belongs to its respective clusters\n"
     ]
    }
   ],
   "source": [
    "#Now we get the feature clusters\n",
    "dep_matrix = 'linear' # linear correlation base dependence matric\n",
    "# n_cluster is set to None for getting the Optimal Number of Clusters using ONC Algorithm\n",
    "clusters = get_feature_clusters(X,dependence_metric=dep_matrix,distance_metric=None,linkage_method=None,n_clusters=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['I_2', 'R_7', 'R_12', 'R_24'],\n",
       " ['I_1', 'R_5', 'R_13', 'R_15', 'R_16', 'R_18', 'R_27'],\n",
       " ['I_3', 'R_2', 'R_3', 'R_4', 'R_6', 'R_20', 'R_22', 'R_25', 'R_28', 'R_29'],\n",
       " ['I_4', 'R_0', 'R_8', 'R_11', 'R_19'],\n",
       " ['I_0', 'R_1', 'R_9', 'R_10', 'R_14', 'R_17', 'R_21', 'R_23', 'R_26'],\n",
       " ['N_0', 'N_1', 'N_2', 'N_3', 'N_4']]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clusters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see that algorithm have not detected any features with low silhoutte score. So, there is no need replace the features with their residuals*. Now, that we have identified the number clusters (six in this case) and composition of features with in each cluster, we can move to the next step.\n",
    "<br> ( *This will be discussed in the later part of this notebook)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step - 2 : Clustered Importance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clustered Feature Importance can be implemented by simply passing the feature clusters obtained in Step-1 to the **clustered_subsets** argument of the MDI or MDA feature importance algorithm. We can apply MDI and MDA on groups of similar features, rather than on individual features and obtain the importance of the cluster as whole instead of individual features. This way we can anlayse how mutually dissimilar clusters interact with model and possibly isolate the noisy/non-infomative clusters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, log_loss\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.model_selection._split import KFold\n",
    "#for out of sample score\n",
    "from mlfinlab.cross_validation import  ml_cross_val_score\n",
    "#Import MDI and MDA\n",
    "from mlfinlab.feature_importance import mean_decrease_impurity, mean_decrease_accuracy ,plot_feature_importance\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Setup for feature importance algorithm \n",
    "#We define a classifier \n",
    "clf_base = DecisionTreeClassifier(criterion='entropy',max_features=1, class_weight='balanced',min_weight_fraction_leaf=0)\n",
    "clf = BaggingClassifier(base_estimator=clf_base,n_estimators=1000, max_features=1.,max_samples=1.,oob_score=True,n_jobs=-1)\n",
    "#fit the classifier\n",
    "fit = clf.fit(X,y)\n",
    "#setting up cross-validation generator\n",
    "#use Purged K-Fold generator while using it on real financial dataset to avoid leakage\n",
    "cvGen = KFold(n_splits=10)\n",
    "oos_score = ml_cross_val_score(clf, X, y, cv_gen=cvGen, sample_weight_train=None, scoring=log_loss).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clustered MDI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We compute the clustered MDI as the sum of the MDI values of the features that constitute that cluster. If there is one feature per cluster, then MDI and clustered MDI are the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "clustered_mdi_imp = mean_decrease_impurity(clf,X.columns,clustered_subsets=clusters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHiCAYAAAA9NBIoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debwkVX3H/c8PGJB1kEXCPioKyOJVr2BiUAQVjBJJ3HABBkEwwkPQ4BqMRh2jEoVHY0YwICgqBBVFcCMoRhJQZ2RgNEQWAZlnNMgyAyMjwvB7/qi60NP0vtS93ffzfr36Nd1Vp+qc7up759yqU+cbmYkkSZKGZ53pboAkSdK4s8MlSZI0ZHa4JEmShswOlyRJ0pDZ4ZIkSRoyO1ySJElDZodLs1JEfCYi3jvd7ZAkzQ52uNSViLg1IlZHxKqax3Z97nP/iFg2qDZ2IjPfnJkfrLLOZiLinIj40HS3o52I2DwiFkbEbyPi/ohYGhFHNSg3v1x3f1l2YURsXrP+/RHxYM335/qIeEWLetePiI9HxLKy/C0Rcdqw3menImKDiDg7Iu4t3+fb2pQ9LSKWR8Q9EfGvETGnZv0WEXFRRPw+Im6LiNfVrHtP3c/b6oh4OCK2KtdvHxHfiIi7y8/ozV2+j0Ed183Lz+O3EXFfRNwQEe/spi0zTUQcGBH/W77nH0TEzi3K/iAifld+H66NiJfXrHtpRFwZESvKz+ezEbFpzfqW36WIOCYibiqP/3f6/Z2r6WGHS704JDM3qXksn87GRMR601l/PyJi3eluQyciYn3gP4CdgT8F5gJvBz5S+59DRPwd8NFy3VzgOeU2l5X7mHLB1PcHOAk4LyK2aVL9u4FJYB9gU+AFwDUDfHu9fofeDzyF4v29AHhHRBzcpOy7KN7DnsBTgWcCp9Ss/zTwR2Ab4PXAwojYAyAzP1z780bx+V6RmXeW254H3FJu+1LgwxHxgk7ewICP62nAJsDuZZm/BG7upB2DNojfCWWH9mvAe4EtgEXABS02+Vtg28zcDDiW4ju9bbluLvAhYDuKz2cH4NSabd9Pk+9SRDwf+DDw8rIdtwBf7vf9aRpkpg8fHT+AW4EXNln3HOC/gRXAtcD+NeuOAq4H7gN+BRxXLt8YWA08DKwqH9sB5wAfqtl+f2BZXTveCVwHPACsV273VeB3FL+UTmzxPh7Z/9S+gXcAdwC/AQ4F/gK4AbgbeE/Ntu8HvkLxy/c+4GfA02vW7w5cUX4OvwD+sq7ehcC3gN9T/GJ+kOI/21XAN8ty76L4z+o+4H+Av6rZx3zgSuCfgXvK9/qSmvVbAJ8Dlpfrv16z7mXAkrJt/w3s3eFxP7r8bDauW/6ast2blY9VwKvrymxSbvvGms/vvLoydwB/1qTuS4CTWrRtR4r/GH8H3AX8S7l8HYpOzW3l/j8PzC3XzQOyfF+/Bv6z3Xe4Qb3/H/DimtcfBM5vUnYR8Kqa168Dbq/5Gfgj8NSa9V8APtJgP1F+L46s+WwT2LqmzJnAF2peXwe8roLj+nPg0A6/T4+j6CjeVX7WPwW26eD7+ybgJoqfyYuB7WrWJXA8cCNwS7lsN+Cysvwv699DmzYeC/x3zeup31W7dbDtPsAfgH2arP9rYGkn3yWKn/NP16zbrnyvT+70vfiYGQ/PcGkgImJ74FKKv+K2AE4GvhoRW5dF7qD4z34zis7XaRHxzMz8PfASYHl2f8bstRR/0W9O0WH7JsV/ktsDBwInRcRBHe7rTyj+E9ge+Afgs8AbgGcB+wH/EBFPqin/cuDC8r1+Cfh6RMwpLxN9E/ge8ATg/wG+GBG71mz7OmABxdmazwNfBD5WvvdDyjI3l/XOBf6Rtf9aBtiX4j+QrYCPAWdFRJTrvgBsBOxRtuE0gIh4JnA2cBywJXAGcHFEbNDB5/Mi4Nvl8ar11fJz+1Pgz8rnX6stkJmrgG+X+1hLFF4KrE/RsWzkauBtEfGWiNir5n1OnSG8hKJTNY/i+J1frp5fPl4APImig/Avdft+PkUH+aB23+GIeFdEXFI+fzzFf3zX1uzrWorPvJEoH7Wvd4iIuRRnvNZk5g0d7Gs/ijNZX63ZT+2/U8/3nHqRmXtn5peatGuQx/VqYEFEHBURT2lS35QjKb7bO1J8F99M0ZmB5t/fA4B/Al4NbEtxzM9nbYdS/Gw8LSI2puhsfancz2uBf506cxgRr4uI61q0cQ9qjm/5Gd1M82NMRFwSEX8AfkzxR9eiJkWfR/HHWCffpUbfHag5xhoR093j8zFaD4ozS6so/ipdQfnXJ8XZpi/Ulf0u5V/iDfbzdeBvy+f7U3P2qlx2Du3PcL2x5vW+wK/r9vFu4HNN6n9k/+W+VwPrlq83pfgLct+a8osp/3qnOENzdc26dSjOiu1XPn4LrFOz/svA+2vq/Xyr99qkvUuAl5fP5wM31azbqGzvn1D8R/Qw8PgG+1gIfLBu2S+B53dw3P+DBmdcynW/pbgM9gbgt03KfAS4rObz+2P5/bkfWAO8o0Xd61KcufgvirOZy3n0DM+fUpzZWq/BdpcDb6l5vSvF2cT1ePQM15Nq1nf8HaboKCTwuJplLwJubfIePlS2f+vyOP243H7bqe9MXfk3UVw2rN/PWcA5dcuuBD5F0Sl6JuXZnHbHdAjHdUPgPRQ/Kw9SnIl6SZPt3kiDM6xtvr9nUfxhMvV6k7KeeeXrBA6oWf8a4Ed1+zgDeF+Hn81Z9Z9NeQznt9luDsUfkW9tsv5FFGfuntrJd4nij8c7gb3Lz/iM8jN6bSfvw8fMeXiGS704NDM3Lx+Hlst2Bl5VDgpdERErgD+n+AVKRLwkIq4uB/auoLhct1Wf7bi95vnOwHZ19b+H4mxAJ+7KzDXl86m/tP+vZv1qil/wj6k7Mx+muCS5Xfm4vVw25TaKMy+N2t1QRBwREUtq3suerP15/bam/vvLp5tQ/PK+OzPvabDbnYG/q/uMdizb3M6dlMeyrp3rle26s3xs1WT8zLbl+in/Xn5/NgKeDBwREcc1qjgz12TmpzPzuRRnMxcAZ0fE7mX7b8vMhxpsuh3FZz/lNorOVu13ov471PQ7XGdV+e9mNcs2o7gE3MgCinFnSyg6Gl+n6CzcwaOX7mo9Zl8RsSHwKuDcurKvB55YvpeFFGdMO70JZWDHNTNXZzHe7FkUZ63+HbgwIrZosN0XKDqz50dxI8HHyrPDrb6/ax3PLM6w3UXzn62dgX3rjufrKTq89e93p6i5MaFc3NFxqZeZD2bmtynOmv5lXT3PoTjj9sp89Ixmy+9SZl4OvI/irONtFH9s3kfnx1gzhB0uDcrtFGcHNq95bJyZHykvWX2VYizCNpm5OcUYpqlT49lgf7+nOHMz5TG/JOu2u51i3EZt/Ztm5l/0/c4a23HqSUSsQzEIdnn52LFcNmUnijEajdr9mNflnVCfBU4Atiw/r5+z9mWFZm4Htoiau8fq1i2o+4w2ysxOBuD+B/CS8jJNrVdQnHW6GriqfP7Xde9nY4q/+C9vtOPMvJXi0tQhjdbXlV2dmZ+mOEPwtPI97dSkM7Cc4j/dKTsBD7F2R7r+O9TwO9ygHfdQnNV8es3ip1NeJmrS7hMyc/vMfBJFR2Fx2cm/AViv7jJco339NcXZqyvq9n1bZr4sM7fOzH0pOjs/adSOBoZyXDPzXoqB3htTdAbr1z+Ymf+YmU+juGT5MuAIWn9/1zqeZf1b0vxn63bgh3XHc5PM/JsG7fl1rn1jAhSf/yPHt6zvyTQ5xg2sV5af2v4ZFOPO3lh2oqbqbvtdKv/geEpmPoHid+l6FL8TNEqm+xSbj9F60GTQPEUH5LfAQRSXgB5HcaluB4pLdGsoxssExS/p+3n0kt5uFGeQ5tbs703A/1KMpfkTil/89ZcUX1jzel2KSxnvpDjtvi7FWaFnN3kf51A3aL5m3XoUv7jn1Sy7EnhD+fz9FGcn/ros+7ayPXMoxiLdTDHofU657/soB9rS4PIhxWWZL9W8fhrFgNtdy/dxFEVH4Zhy/Xzgyrp9JLBL+fxSir+iH1+24Xnl8kmK/4T2LY/DxhRj4Dbt4LhvQHFzwLcoLsfNKY/1/wFvryn3jnLZwWWZeeU2PwM2qPn8zqvZZgdgKfDRJnWfVH6OG5af95EUHYAnlZ/PtRSd+Y0pvnfPLbc7hmIA9RMpzv59ZapeHr2kuF5NPU2/w03a9RHgh+XnvBvFf5oHNym7PcUZmqAYmH87aw+SPp/i0vPGwHOBlcAedfv4HvCBBvveneJnbH2Ky393svYg+ltpchlswMf1vcCzy3Y8Dvh7io7xJg3qfQGwV/k5b1Eew/ltvr8HUlw+nijb/f9S83NAzc9A+XpTijNCh5f7mVO2b/cOf9dtXR6HV5Tv56PUDCWoK7sbxe+1Dct63kBx2fyZ5fo9y8/vNd1+l8q69yy/OztRdLg/3O3vbh/T/5j2BvgYrQet71Lct/ylcXf5i/FSYKdy3fHlL5wVFJcTzmftMVpn8+gdS9uVv2QuAO6luMvqrbTocJXLtqP4T+u35S/6q1u09Rz663DV3qV4zdQv1nL9HuXnsJLH3mH4SL01y57Co3cOTo2JW1B+jncCnyj312mHawuKy07/V34OX6spdzDFHWEryl/qF1J2uCjOMr2n0edVs98zyv2upvgL/JgG5Y6m+Ot7dVn2DGrG5PBoh3XqrtTfAJ8BNmpS73EUnemVZbt/ArysZv1OFJfo7io/r0+Wy9ehuAHidorv43lT7aBBh6uD7/B7KAaYT5XdgOJ7e2/5Pt9W16ZVNds+j+I7ez/FuLnXN/hsv05xZvfX1N1VSNFhe4iaDkXNupPKtv6e4ns6WbNufWo6/EM+rqeU6+/l0TNxze48fW35Ofy+3Ncnp44Frb+/b6b4g+ZuipsldqhZt1aHq1y2a3kMp+5g/T4wUa57PfCLNr/vXkjxh9/q8v3U/k74DPCZ8vnuFOPy7uPRuy5rf+4/x9p3Yq+qrbvNd2lzit+Bv6f43fZPlONNfYzWI8oDKqlDEfF+il/sb5jutkitRMSfA8dn5munuy3SbDeyE0ZKklrLzCspznpJmmYOmpckSRoyLylKkiQNmWe4JEmShswOlyRJ0pDN6EHzW221Vc6bN2+6myFJktTW4sWL78zMrRutm9Edrnnz5rFoUbPsT0mSpJkjIm5rts5LipIkSUPW9gxXRKyhiN1YD7gFODwzVzQpO0ERnroZRZTLgsy8oFz3I4qoBYAnAD/JR4OPG7rvPrjiis7eiCRJ0pT995/uFqytk0uKqzNzAiAizqWIaFnQpOz9wBGZeWNEbAcsjojvZuaKzNxvqlBEfBX4Rp9tlyRJeoyJk/YvQpFgxpy56faS4lUUmV4NZeYNmXlj+Xw5cAdFAOgjImJT4ACK3LDHiIhjI2JRRCxaufJ3XTZPkiRp5ul40HxErEuR1n5Wh+X3oQhOvblu1V8Bl2fmvY22y8wzgTMBdt110llZJUlSV5acfsWMu6TYyRmuDSNiCUXS+hbAZe02iIhtgS8AR2Xmw3WrXwt8uduGSpIkjaqOx3BFxFzgEooxXJ9sVjgiNgMuBU7JzKvr1m0J7ENxlqutTTedeYPeJEmSutXxGK7MXAmcCJwcEXMalYmI9YGLgM9n5oUNirwKuCQz/9BLYyVJkkZRV4PmM/Ma4FrgsCZFXg08D5gfEUvKx0TN+sPwcqIkSZpl2l5SzMxN6l4f0qLsecB5Ldbv303jJEmSxoEzzUuSJA1ZT1mKEbEXxV2ItR4A9qg/I1a33XeA5wBXZubLeqlbkiRp1PTU4crMpcBE/fKIWNVm01OBjYDjOqnHaB9JktSLmTbLQU8drl5l5uURsX+VdUqSpNllHKJ9hs5oH0mSNG4qPcPVCaN9JElSP0Y12keSJEl9mHFnuGoZ7SNJksZBpR2uiPgRsBuwSUQsA47OzO9W2QZJkqSqDbTD1WoOrnL9foOsT5IkaRQ4hkuSJGnIBn5Jsdks9Jm576DrkiRJGgV9dbgiYg2wtNzPLcDhzWahrysP8OvM/MtW+3emeUmS1IuZdtNdv5cUV2fmRGbuCdwNHN9h+Yl2nS1JkqRxMchLilcBew9wf5IkSV0b22ifiFgXOBC4uE3Rx5WxPVdHxKFN9mW0jyRJGiuR2Xt6Ts2YrHnAYuDFmbmmRfntMnN5RDwJ+D5wYGbe3Kz8rrtO5hlnLOq5fZIkaXaajjFcEbE4MycbrRvIGC5gZ2B92ozhyszl5b+/Aq4AntFn/ZIkSTPeQMZwZebKiDgR+EZELMzMB+vLRMTjgfsz84GI2Ap4LvCxVvs12keSJI2DgU18mpnXANcChzUpsjuwKCKuBX4AfCQz/2dQ9UuSJM1UfZ3hqo/yycxDWpT9b2CvfuqTJEkaRUb7SJIkDZnRPpIkSUPWtsPVJL5nRYtNTqWYJuLKzHxZzX7OAiaBAG4A5mfmqlZ1G+0jSZJ6MdNuuuvkkmK38T2nAoc3WP7WzHx6Zu4N/Bo4obumSpIkjaZuLym2je/JzMsjYv8Gy+8FiIgANgR6n3FVkiSpiZGO9ukivqfVPj4H/BbYDfhUkzJG+0iSpLHSSYdrw4hYAtwFbAFc1mtlmXkUsB1wPfCaJmXOzMzJzJycO3frXquSJEmz1JLTryjObM2Qs1vQxRguOozvaafMWrwAeEU/+5EkSRoVHY/h6iS+p5ly3NaTM/Om8vkhwP+2285oH0mSNA66mvi0g/geIuJHwIXAgRGxLCIOopgK4tyIWEoxxcS2wAd6brUkSdIIaXuGq5v4nnL9fk1WPbeLdkmSJI0No30kSZKGrKdoH+N7JEmSOtdrtM9SYKJJ+e8Az+Gx0T7nAM8HVpaL5mfmklZ1G+0jSZJ6MdNuuqsy2gfg7eW+Jtp1tiRJksZFZdE+kiRJVZj10T7Agoi4LiJOi4gNmtRjtI8kSRorVUb7vJsiQ/HZ5X7e2aiQ0T6SJKkfszraJzN/k4UHgM8B+/SyH0mSpFFTSbQPQERsm5m/KaN9DgV+3m4bo30kSdI4qCraB+CLNdE+WwEf6q3JkiRJo6WyaJ/MPKC7pkmSJI0Ho30kSZKGzGgfSZKkIasy2icoxm29ClgDLMzMT7aq22gfSZLUi5l2012V0T7zgR2B3TJzd+D8bhoqSZI0qqqM9vkb4HWZ+XBZ7o4u65YkSWprtkf7PBl4TRnb8+2IeEqTeoz2kSRJY6XKaJ8NgD9k5iTwWeDsRoWM9pEkSf2YidE+nVxSXJ2ZExExF7iEYgxXy8HuTSwDvlo+v4gi3qclZ5qXJEnjoONLipm5EjgRODki5vRQ19eBqclPnw/c0MM+JEmSRk6V0T4fAV5Rxvv8E3BMb02WJEkaLVVG+6wAXtpV6yRJksaA0T6SJElDZrSPJEnSkFUZ7XMgxSz06wCrgPmZeVOruo32kSRJvZhpsxxUGe2zEHh9Zk4AXwJO6aqlkiRJI6rKaJ8ENiufzwWWd1m3JElSWzMx2qfjDldNtM9ZPdZ1DPCtiFgN3Etx2bFRPccCxwJss81OPVYlSZI0c1QZ7fNW4C8ycweKWeY/0aiQ0T6SJKkfszbaJyK2Bp6emT8uF10AfKfddkb7SJKkcVBVtM89wNyIeGr5+kXA9V3uQ5IkaSR1NWg+M6+JiKlon/p5uIBHon12AzaJiGXA0Zn53Yh4E/DViHiYogP2xv6aLkmSNBqqjPa5CLioq9ZJkiSNAaN9JEmShsxoH0mSpCHrqcM1Fe1TF/tze0Rsnpkrmm0XER8FXlq+/GBmXtCqHqN9JGlwvOtbmj79XlLsOPYnIl4KPJMig3Ff4O0RsVmz8pKkwZk4aX97XNI0GuQYrquA7Vusfxrww8x8KDN/D1wLHFxfKCKOjYhFEbFo5crfDbB5kiRJ02MgHa6a2J+LWxS7FnhJRGwUEVsBLwB2rC/kTPOSNHiPzLwtaVr0NIarxlTszzxgMS1ifzLzexHxbOC/gd9RnBF7qM/6JUmSZrx+O1xdxf5k5gJgAUBEfAm4sdXOjfaRJEnjYCCXFDuJ/YmIdSNiy/L53sDewPcGUb8kSdJM1u8Zrkd0EPszB/hRRADcC7whM72kKEmSxl5fHa5uYn8y8w8UdypKkiTNKkb7SJIkDdnALilOMfZHkiRpbW07XHXxPbcAh7eK7wFOpZgm4srMfFnNfn4EbFq+fALwk8w8tFXdRvtI0uB417c0fTq5pNhxfE/pVODw+oWZuV+5nwmKObi+1nVrJUk9MdpHml7djuFqF99DZl4O3NdsfURsChwAfL3JeqN9JEnSWOm4w9VhfE8n/gq4PDPvbbTSaB9JGjyjfaTp1UmHayq+5y5gC1rE93TotcCX+9yHJEnSyOjkLsWu4ntaKWea34fiLFdbRvtIkqRx0PElxU7iezrwKuCSchJUSZKkWaGrQfOZeQ0wFd/TUDn9w4XAgRGxLCIOqll9GF5OlCRJs0zbS4rdxPeU6/drsW7/jlsmSZI0Joz2kSRJGrKeon1q4nueDkyNx/oD8MRWs9BHxHeA51A3C30zzjQvSYPjTUjS9OnpDFdmLi1njP99Zm6YmRtSzM/V0yz0kiRJ42yQlxT7noVekjQcRvtI02sgHa4BzkJvtI8kSRo7/Xa4Bj0LvdE+kjQERvtI06vfDtfqcizXzsD6tB/DJUmSNOv0dJdivcxcGREnAt+IiIWZ+eAg9mu0jyRJGgcDGzQ/gFnoJUmSxlJfZ7gGOQu9JEnSuHKmeUmSpCEbyBiuWjWz0Nd6IDP3HXRdkiRJo6Bthysi1gBLy7K3AIe3iu+hmE1+HnXxPRHxROB8iukjflbu54+t6jbaR5IGx5uQpOnTySXF1Zk5kZl7AnfTe3zPR4HTMvMpwD3A0V21VJIkaUR1O4arp/ieiAjgAOAr5aJzgUO7rFuS1COjfaTp1XGHq8/4ni2BFZn5UPl6GU06bkb7SJKkcdNJh2sQ8T3RYFk2Kmi0jyQNntE+0vTqeAwX/cX33AlsHhFTg/R3AJb3sB9JkqSR0/G0EP3E92RmRsQPgFdS3Kl4JPCNdtsZ7SNJksZBV4Pm+4zveSfwtoi4iWJM11m9NVmSJGm0tD3DNaj4nsz8FbBPV62TJEkaA0b7SJIkDVlP0T7G90iSJHWu12ifpcBEk/LfAZ5DXbRPzfpPAUfVX6psxGgfSRocb0KSpk+V0T5ExCSweXdNlCRJGm2VRPvAIzPVnwq8o8s6JUl9MtpHml5VRfsAnABcnJm/aVOP0T6SJGmsVBLtExHbAa8CPtWurNE+kjR4RvtI06uqaJ9nALsAN0XErcBG5QSokiRJY6+qaJ9LgT+Zeh0RqzJzl3bbGe0jSZLGQZXRPpIkSbNSZdE+rfYpSZI0zoz2kSRJGjKjfSRJkoaspw5Xs2ifcjB808uFEXEkcEr58kOZeW6reoz2kaTB8SYkafr01OHqRURsAbwPmAQSWBwRF2fmPVW1QZIkaTpUOYbrIOCyzLy77GRdBhxcYf2SNGsZ7SNNryo7XNsDt9e8XkaDXEajfSRJ0ripssMVDZblYxYY7SNJA2e0jzS9quxwLQN2rHm9A7C8wvolSZKmRWWD5oHvAh+OiMeXr18MvLvVBkb7SJKkcVBZhysz746IDwI/LRd9IDPvrqp+SZKk6TLQDle7yJ7MPBs4e5B1SpIkzXRG+0iSJA3ZwC8pGvsjSZK0trYdrohYAywty94CHJ6ZK5qUnQAWAnOANcCCzLygXHcCcBLwZGDrzLyzXd1G+0jS4HgTkjR9OrmkuDozJzJzT+Bu4PgWZe8HjsjMPShmkT89IjYv1/0X8ELgtn4aLEmSNGq6vaR4FbB3s5WZeUPN8+URcQewNbAiM68BiGg0/6kkaZgmTtofNsfLBtI06XjQfESsCxwIXNxh+X2A9YGbu2mQ0T6SJGncdNLh2jAilgB3AVtQhE63FBHbUgycPyozH+6mQUb7SNLgGe0jTa9OLimuzsyJiJgLXEIxhuuTzQpHxGbApcApmXl1P41zpnlJkjQOOr6kmJkrgROBkyNiTqMyEbE+cBHw+cy8cDBNlCRJGm1dTXxaDny/FjisSZFXA88D5kfEkvIxARARJ0bEMorQ6usi4t/6aLckSdLIiMyc7jY0NTk5mYsWLZruZkiSJLUVEYszc7LROqN9JEmShqynaB/jeyRJkjrXa7TPUmCiQdmpaJ/NeGy0TwAfAl5VrluYmU3vdgSjfSRpkLzrW5o+VUb7zAd2BHbLzN2B83tvtiRJ0uioLNoH+BvgdVMToWbmHd03V5LUC6N9pOlVZbTPk4HXlLE9346IpzTZzmgfSZI0VqqM9tkA+EN5u+RngbMbbWu0jyQNntE+0vSqMtpnGfDV8vlFwOfaVWy0jyRJGgdVRvt8HTigfP584AYkSZJmgcqifYCPAK+IiKXAPwHH9NhmSZKkkdL2kmJmblL3+pAWZc8DzmuybgXw0m4bKEmSNOqM9pEkSRoyo30kSZKGrMpon7OASSAoBszPz8xVreo22keSBse7vqXpU2W0z1sz8+mZuTfwa+CEfhouSercxEn72+OSplG3Y7iuArZvtjIzb8jMG8vny4GpaB8y8154JMR6QyAb7cOZ5iVJ0ripMtqHiPgc8FtgN+BTjbZzpnlJGjxnmpemV5XRPmTmUcB2wPXAa3pqsSRJ0oipMtoHgMxcExEXAG+nTbyP0T6SJGkcVBLtE4Vdpp4DhwD/20/DJUmSRkVV0T4BnFvG+iwFtgU+0HuzJUmSRkdl0T7Ac7trmiRJ0ngw2keSJGnI2na4ImJNeWnw5xHxzYjYPCL2qrlkOPX4cVn+OxGxIiIuqdvPARHxs3I/50ZET7FCkiRJo6bjuxQBIuJc4PjMXECDaJ/SqcBGwHFTCyJiHeBc4MDMvCEiPgAcCZzVqmKjfSTV885lSaNooDPNA2Tm5cB9dYu3pAi3vqF8fRnwii7rljTLPRJPY69L0ogZ2kzzde4E5kTEZPn6lcCOTeox2keSJI2Vocw0Xy8zk2IqidMi4icUZ8AealLWaB9JDT0ST+NYA0kjppMO19QYrp0pshGP76WizLwqM/fLzH2A/wRu7GU/kiRJo6bjO3O60FIAABwcSURBVAUzc2VEnAh8IyIWZuaD3VQUEU/IzDsiYgPgncCCdtsY7SNJksbBoGeaJyJ+BFwIHBgRyyLioHLV2yPieuA64JuZ+f0e2yxJkjRSBjrTfLl+vybL304RWC1JkjSrONO8JEnSkPU023tE7AV8oW7xA5m5b/9NkiRJGi9tO1wRsQZYWpa9BTg8M5fSZKb5iPgO8Bzgysx8Wc3yE4CTgCcDW2fmne3qdqZ5SfW8kUbSKOp4WojM3BO4m/bTQpwKHN5g+X8BLwRu666JkiRJo63bS4pXAXu3KpCZl0fE/g2WXwMQEV1WKUmFiZP2h83LF57+ljRCqor26ZjRPpIkadx0coZrKtpnHrCYHqJ9upGZZwJnAuy662QOsy5Jo2XJ6Vc4hkvSSKos2keSJGm2qizapxdG+0iSpHFQWbRPRJwYEcuAHYDrIuLfem+2JEnS6Kgy2ueTwCe7ap0kSdIYMNpHkiRpyIz2kSRJGrKeOlzNon0iYlX9JcgGZTYDrgcuyswTWpU12kdSPW+kkTSKpuOS4geBH05DvZIkSdOipzNcvYqIZwHbAN8BJqusW9LoM9pH0qiq7AxXRKwDfBx4e5tyRvtIkqSxUuUZrrcA38rM21sFWBvtI6kZo30kjaoqO1x/CuwXEW8BNgHWLwfZv6vCNkiSJFWusg5XZr5+6nlEzAcm23W2jPaRJEnjwIlPJUmShmygZ7jazcFVU+4c4JxB1i1JkjRTeYZLkiRpyAY+hsvYH0mSpLX11eGKiDXA0nI/twCHN4v9Kct/DHgpxZm1y4C/zcymUz8Y7SOpnjfSSBpF/V5SXJ2ZE5m5J3A3cHyzghHxZ8Bzgb2BPYFnA8/vs35JkqQZb5CXFK+i6Ew1k8DjgPWBAOYA/zfA+iWNOaN9JI2qgQyaj4h1gQOBi5uVycyrgB8Avykf383M6xvsy2gfSZI0VvrtcG0YEUuAu4AtKMZlNRQRuwC7AzsA2wMHRMTz6stl5pmZOZmZk3Pnbt1n8ySNkyWnX1Gc2fLslqQRM5AxXMDOFJcKm47hAv4KuDozV2XmKuDbwHP6rF+SJGnGG8gYrsxcGREnAt+IiIWZ+WCDYr8G3hQR/0Qxhuv5wOmt9mu0jyRJGgcDm/g0M68BrgUOa1LkK8DNFNNIXAtcm5nfHFT9kiRJM1VfZ7jqo3wy85AWZdcAx/VTnyRJ0igy2keSJGnIjPaRJEkasqqjfY4ETilffigzz221f6N9JNXzRhpJo6jKaJ8tgPcB+wL7AO+LiMf3Wb8kSdKMV2W0z0HAZZl5N0BEXAYcDHx5gG2QNMaM9pE0qiqL9qGYXf72mtfLymX1+zLaR5IkjZV+z3BNRfvMAxbTItqHYrLTevmYBZlnAmcC7Lrr5GPWS5q9lpx+hWO4JI2kKqN9lgE71rzeAVjeZ/2SJEkzXmT2fhIpIlZNTX4aEc8AvgE8uVG0TzlofjHwzHLRz4BnTY3pamRycjIXLVrUc/skSZKqEhGLM3Oy0brKon3KjtUHgZ+Wjw+06mxJkiSNi8qifcr1ZwNn91OnJEnSqDHaR5IkaciM9pEkSRqyth2uJvE9K5qUnQAWAnOANcCCzLygXPdE4HxgC4oB84dn5h9b1W20jzR+nNZB0mzUySXFjuN7gPuBIzJzD4pZ5E+PiKl5oT8KnJaZTwHuAY7uo92SJEkjo9sxXFfRYHb4KZl5Q2beWD5fDtwBbB0RARwAfKUsei5waPfNlTTKJk7af7qbIEnTouMOV4fxPbXl96GYDPVmYEtgRWY+VK5uGOtTbme0jyRJGiuddLim4nvuohh/1Sq+B4CI2JZi4PxRmfkwHcb6QBHtk5mTmTk5d+7WHTRP0qhYcvoV090ESZoWndyluDozJyJiLnAJxRiuTzYrHBGbAZcCp2Tm1eXiO4HNI2K98ixXR7E+m27qAFtJkjT6Or6kmJkrgROBkyNiTqMyEbE+cBHw+cy8sGbbBH4AvLJcdCRFDJAkSdLY62rQfLv4HuDVwPOA+RGxpHxMlOveCbwtIm6iGNN1Vo9tliRJGiltLyl2E9+TmecB5zVZ9ytgn24bKEmSNOqM9pEkSRqynqJ9jO+RJEnqXE8drsxcCkzUL4+IVfWXIGvW7Qx8DViXIvrnU5n5mVb1GO0jjR/vPJY0Gw08vLqF3wB/lpkPRMQmwM8j4uJyRnpJkqSxVdkYrsz8Y2Y+UL7coMq6Jc0MRvtImq0q7fRExI4RcR1wO/DRRme3jPaRJEnjptIOV2benpl7A7sAR0bENg3KGO0jjSmjfSTNVlWO4XpEZi6PiF8A+wFfaVbOaB9JkjQOKjvDFRE7RMSG5fPHA88FfllV/ZIkSdOlyjNcuwMfj4gEAvjncnoJSZKksTbQDlezObjKdZcBew+yPkmSpFHg1AySJElDNvBLisb+SJIkra1thysi1gBLy7K3AIdn5oomZSeAhRTRPWuABZl5QbnuHOD5wMqy+PzMXNKqbqN9pPHjnceSZqNOLimuzsyJzNwTuBs4vkXZ+4EjMnMP4GDg9IjYvGb928t9TbTrbEkaP840L2m26nYM11XA9s1WZuYNmXlj+Xw5cAfQ1eylzjQvSZLGTccdrohYFzgQuLjD8vsA6wM31yxeEBHXRcRpEbFBo+2caV4aX840L2m26qTDtWFELAHuArYALmu3QURsSzFw/qjMfLhc/G5gN+DZ5X7e2VOLJUmSRkwndymuzsyJiJgLXEIxhuuTzQpHxGbApcApmXn11PLM/E359IGI+BxwcruKjfaRJEnjoONLipm5EjgRODki5jQqExHrAxcBn8/MC+vWbVv+G8ChwM97bbQkSdIo6WrQfGZeA1wLHNakyKuB5wHzI2JJ+Zgo130xIpZSTDGxFfChHtssSZI0UtpeUqyP68nMQ1qUPQ84r8m6A7punSRJ0hgw2keSJGnIeupwRcRe5eXCjIjV5eOeuklO67eZiIirIuIX5dQQr+m92ZIkSaOjpyzFzFwKTETEqqlLjhFxLsUdjAuabDY1C/2NEbEdsDgivtssJgiM9pHGkXceS5qNBnlJceiz0EsabUb7SJqtBtLhGtAs9FPrjPaRJEljpd8O16BmoX+E0T7S+DLaR9Js1W+Ha3VmTgA7U5yxOr5V4Waz0EuSJI2zngbN18vMlRFxIvCNiFiYmQ/Wl2k1C30zRvtIkqRxMLBB833OQi9JkjS2+jrDNahZ6CVJksaZM81LkiQN2UDGcNWKiL0o7kKs9UBm7jvouiRJkkZBXx2uiFgDLC33cwtw+NQs9A3K7gx8DVgXmAN8KjM/02r/zjQvjR9vhJE0Gw1kWojM3BO4m9bTQvwG+LNyGol9gXeVET+SJEljrcponz9m5gPlyw0GXLekEWC0j6TZqtJon4jYMSKuA24HPlpmKtaXMdpHkiSNlUqjfTLz9szcG9gFODIitmlQxmgfaUwZ7SNptqo02mdKeWbrF8B+fdYvSZI041UZ7bMDcFdmro6IxwPPBT7Rar9G+0iSpHFQZbTP7sCPI+Ja4IfAP5dTSEiSJI21KqN9LgP27qc+SZKkUeTUDJIkSUNmtI8kSdKQVRbtU7PNZsD1wEWZeUKr/RvtI40fb4SRNBtVGe0z5YMUg+YlSZJmhcqifQAi4lnANsD3BlivpBFhtI+k2aqyaJ+IWAf4OPD2Nvsy2keSJI2VKqN93gJ8KzNvb7VDo32k8WW0j6TZqsponz8FToiIW4F/Bo6IiI/0Wb8kSdKMV1m0T2a+fup5RMwHJjPzXa32a7SPJEkaB1VG+0iSJM1KlUX71JU7Bzinn7olSZJGhdE+kiRJQ2a0jyRJ0pBVGu0TETsB/wbsCCTwF5l5a7P9G+0jjR9vhJE0G1Ud7fN54NTM3B3YB7ijz/olSZJmvMqifSLiacB6mXkZQGauysz7B1i/pBnOaB9Js1Vl0T7AU4EVEfG1iLgmIk4tt6vfl9E+kiRprFQZ7bMesB9wMvBs4EnA/PpCRvtI48toH0mzVZXRPsuAazLzV5n5EPB14Jl91i9JkjTjVRbtA/wUeHxEbJ2ZvwMOABa12q/RPpIkaRxUFu2TmWsoLideHhFLgQA+O6j6JUmSZqpKo33KOxT37qdOSZKkUWO0jyRJ0pAZ7SNJkjRkbTtcTeJ7VjQpOwEsBOYAa4AFmXlBue5A4FSKs2qrgPmZeVOruo32kcaPN8JImo06uaTYTXzP/cARmbkHcDBwekRsXq5bCLy+nEbiS8ApfbRbkiRpZHQ7hqtlfE9m3pCZN5bPl1NkJU7NXprAZuXzucDyLuuWNOKM9pE0W3U8hqsmvuesDsvvQzEZ6s3lomOAb0XEauBe4DlNtjsWOBZgm2126rR5kiRJM1YnZ7i6ie8BICK2pRg4f1RmPlwufivwF5m5A/A54BONtjXaRxpfRvtImq06OcO1OjMnImIucAnFGK5PNiscEZsBlwKnZObV5bKtgadn5o/LYhcA32lXsTPNS5KkcdDxGK7MXAmcCJwcEXMalYmI9YGLgM9n5oU1q+4B5kbEU8vXLwKu763JkiRJo6Wrebgy85qImIrvqZ9rC+DVwPOALSNifrlsfmYuiYg3AV+NiIcpOmBv7L3ZkiRJoyMyc7rb0NTk5GQuWtQy31qSJGlGiIjFmTnZaJ3RPpIkSUPWU7SP8T2SJEmd6zXaZykw0aDsVLTPZjw22ueLwCTwIPAT4LjMfLBV3Ub7SOPHO48lzUZVRvt8EdgN2AvYkGIiVEmSpLFXWbRPZn4rSxRnuHborcmSRpXRPpJmq447XDXRPhd3WL4+2mdq+RzgcJpMfBoRx0bEoohYtHLl7zptniRJ0oxVZbTPlH8F/jMzf9RoW6N9pPFltI+k2aqSaJ+ade+juMR4XCeNM9pHkiSNg6qifYiIY4CDgNc2OOslSZI0troaNJ+Z1wBT0T6NTEX7zI+IJeVjavqIzwDbAFeVy/+h10ZLkiSNkraXFDNzk7rXh7Qoex5wXpN1PU2yKkmSNOqM9pEkSRoyo30kSZKGrMponwOAf6aYm2sxcHRmPtSqbqN9pPHjnceSZqNKon0iYh3gXOCwcj+3AUf22XZJkqSRUFW0z5YUlxxvKIteBryi++ZKGmVG+0iaraqK9rkTmBMRk+XqVwI7NtnOaB9JkjRWKon2KQOrDwNOi4ifAPcBDcdvGe0jjS+jfSTNVpVF+2TmVcB+ZZkXA09tV7HRPpIkaRxUGe3zhPLfDYB3Usw8L0mSNPaqjPZ5e0RcD1wHfDMzv99royVJkkZJFMOrZqbJyclctGjRdDdDkiSprYhYnJmTjdYZ7SNJkjRkRvtIkiQNWWXRPjVlPkUxXcQm7eo22kcaTd5dLElr63haCICIOJdiWogFTcpORfvcGBHbAYsj4ruZuaLcfhLYfADtljQDPTKT/NRPuX8xSRJQXbTP1Ez1pwLvaFWBM81LkqRx0/EYrppon7M6LF8b7QNwAnBxZv4mIppul5lnAmcC7Lrr5My9hVLSY0zNJO8lRUlaWycdrqlon3nAYrqL9jkyMx8uLy++Cti/96ZKkiSNpqqifZ4B7ALcVJ7d2igibsrMXVpVbLSPJEkaB5VE+2TmpZn5J5k5LzPnAfe362xJkiSNiyqjfSRJkmaltpcU6+fLysxDWpQ9Dziv231KkiSNM6N9JEmShqynDldE7FVeLsyIWF0+7omIppOaRsQLai4zLomIP0TEob03XZIkaTT0lKU4Fe0TEaumLg+2m4U+M39AGQcUEVsANwHfa1WP0T7SaPLuYklaW08driauAvbusOwrgW9n5v0DrF/SNDPaR5IaG8gYrppZ6C/ucJPDgC832ZfRPpIkaaxEZu/pORGxBljKo7PQvzgz17TZZlvgOmC7zHywVdldd53MM85Y1HP7JE0PLylKmo0iYnFmTjZa1+8ZrtWZOQHsTJGbeHwH27wauKhdZ0uSJGlcDGQMV2aujIgTgW9ExMI2nanXAu/uZL9G+0iSpHEwsHm4OpiFnoiYB+wI/HBQ9UqSJM10fZ3h6mYW+nL9rcD2/dQpSZI0apxpXpIkacgGOQ8XUMxCD3yhbvEDmbnvoOuSJEkaBW07XDVTP6wH3AIcnpkrWmxyKsU0EVdm5stq9vNFYBJ4EPgJcFy7OxWdaV4aTd7sIklr6+SS4urMnMjMPYG7aT/1w6nA4Q2WfxHYDdgL2BA4ppuGSpIkjapuLym2je/JzMsjYv8Gy7819TwifgLs0GXdkmY4o30kqbGOB833EN/TbD9zKM6AfafJeqN9JEnSWOnkDNeGEbGER+N7Luuzzn8F/jMzf9RoZWaeCZwJRbRPn3VJqtCS068AHMMlSfU6HsNFd/E9DUXE+4Ctgbf1ug9JkqRR0/EYri7jex4jIo4BDgIOzMyHO9nGaB9JkjQOupr4tMP4nh8BFwIHRsSyiDioXPUZYBvgqohYEhH/0GObJUmSRkrbM1w9xPfs12T5wCdZlSRJGgVG+0iSJA1ZT2edjO+RJEnqXE8drsxcCkzUL4+IVfWXIGvWTQALgc2ANcCCzLygVT1G+0ijyZtdJGltVY6ruh84IjNvjIjtgMUR8d02uYySJEkjr7IOV2beUPN8eUTcQTEnlx0uaUwY7SNJjU3LoPmI2IdiEtWbG6wz2keSJI2VyBxcek6rMVw1ZbYFrgCOzMyrW5XdddfJPOOMRQNrn6RqOIZL0mwUEYszc7LRukrPcEXEZsClwCntOluSJEnjorIxXBGxPnAR8PnMvLCTbYz2kSRJ46DKM1yvBp4HzC+jfZaUU0VIkiSNtYGe4Wo1fiszzwPOG2R9kiRJo8BoH0mSpCEb+BguY38kSZLW1nOHKyIS+ERm/l35+mRgk8x8Pw1if2q2eyVwIfDszGw554PRPjODNy5IktSffi4pPgD8dURs1ekGEbEpcCLw4z7qlSRJGin9dLgeAs4E3trFNh8EPgb8oY96VTVPcUmS1Jd+B81/Gnh9RMxtVzAingHsmJmXtClntI8kSRorfXW4MvNe4PMUlwmbioh1gNOAv+tgn2dm5mRmTs6du3U/zdOgOJBOkqS+DGJaiNOBo4GNW5TZFNgTuCIibgWeA1wcEQ3zhiRJksZJ39NCZObdEfHvFJ2us5uUWQk8Mrg+Iq4ATm53l6LRPpIkaRwMauLTj1PToZIkSdKjej7DVRvjk5n/B2zUxbb791qvJEnSqDHaR5IkaciGEe3z98Cr6hZfmJkLBl2XJEnSKBhWtM9jOlcR8WbgeGANsAo4NjP/p1UdRvvMDN64IElSf6qM9vlSZu6VmRMUs81/oo+6JUmSRkZl0T7lJKlTNgayj7pVJU9xSZLUl37HcH0auC4iPtZJ4Yg4HngbsD5wQJMyxwLHAmyzzU59Nk+SJGn6VRLtU1P+05n5ZOCdwClNyhjtM9M4kE6SpL4M4i7F04GfAZ/rYpvzgYXtCjnTvCRJGgd9z8OVmXcDU9E+TUXEU2pevhS4sd+6JUmSRsGg5uH6OHBCmzInRMQLgQeBe4AjB1S3JEnSjFZZtE9m/m2vdUmSJI0yo30kSZKGzGgfSZKkIasy2udtwDEUE6b+DnhjZt7Wqg6jfWYG7xSVJKk/VUb7XANMZubewFco4n0kSZLGXpXRPj/IzPvLl1cDO/RRt6rkKS5JkvrS76D5TwOvj4i5XW53NPDtRisi4tiIWBQRi1au/F2fzZMkSZp+lUb7AETEG4BJ4NQm+zTaZ6ZxIJ0kSX2pNNqnnPj074HnZ+YD7cob7SNJksZBldE+zwDOAP4yM+/ot15JkqRRMaiJTz8OtLtb8VRgE+DCiFgSERcPqG5JkqQZrcponxf2WpckSdIoM9pHkiRpyIz2kSRJGrIqo32eR3FH497AYZn5lXZ1GO0zM3inqCRJ/aky2ufXwHzgS33Uqelgj0uSpL5UGe1za2ZeBzzcqpwzzUuSpHEzXdE+TTnT/AzkdV1JkvpSebSPJEnSbFNptE+3jPaRJEnjoLJoH0mSpNmqsmifiHh2RCyjmKPrjIj4xYDqliRJmtGqjPb5KbBDr/VJkiSNKqN9JEmShsxoH0mSpCGrMtpnA4opJJ4F3AW8JjNvbVWH0T4zg3eKSpLUnyqjfY4G7snMXYDTgI/2UbeqZI9LkqS+VBbtA7wcOLd8/hXgwIiI+kJG+0iSpHFTZbTP9sDtAJn5ELAS2LK+kNE+M5DXdSVJ6kuV0T6POZsFZD/1S5IkjYIqo32WATsCyyJiPWAucHerDYz2kSRJ46DKaJ+LgSPL568Evp+ZnuGSJEljr7JoH+AsYMuIuAl4G/CuAdUtSZI0o8VMPskUEfcBv5zudmhotgLunO5GaGg8vuPN4zvePL692TkzG97xN/CZ5gfsl5k5Od2N0HBExCKP7/jy+I43j+948/gOntE+kiRJQzbwDlfZsbJzJUmSVBrUoPlhOXO6G6Ch8viON4/vePP4jjeP74DN6EHzkiRJ42Cmn+GSJEkaedPS4YqIgyPilxFxU0Q8Zj6uiNggIi4o1/84IubVrHt3ufyXEXFQle1WZ3o9vhGxZUT8ICJWRcS/VN1udaaP4/uiiFgcEUvLfw+ouu1qr4/ju09ELCkf10bEX1XddrXXz/+/5fqdyt/RJ1fV5nFReYcrItalCL1+CfA04LUR8bS6YkcD92TmLsBpwEfLbZ8GHAbsARwM/Gu5P80Q/Rxf4A/AewF/kGeoPo/vncAhmbkXRerEF6pptTrV5/H9OTCZmRMUv5/PKGPcNEP0eXynnAZ8e9htHUfTcYZrH+CmzPxVZv4ROB94eV2ZlwPnls+/AhwYEVEuPz8zH8jMW4Cbyv1p5uj5+Gbm7zPzSoqOl2amfo7vNZm5vFz+C+BxEbFBJa1Wp/o5vvdn5kPl8scBDhCeefr5/5eIOBT4FcXPr7o0HR2u7YHba14vK5c1LFP+AK8EtuxwW02vfo6vZr5BHd9XANdk5gNDaqd609fxjYh9I+IXwFLgzTUdMM0MPR/fiNgYeCfwjxW0cyxNR4crGiyr/0uoWZlOttX06uf4aubr+/hGxB4UlymOG2C7NBh9Hd/M/HFm7gE8G3h3RDxuwO1Tf/o5vv8InJaZqwbeqlliOjpcy4Ada17vACxvVqYcAzAXuLvDbTW9+jm+mvn6Or4RsQNwEXBEZt489NaqWwP5+c3M64HfA3sOraXqRT/Hd1/gYxFxK3AS8J6IOGHYDR4n09Hh+inwlIh4YkSsTzEI/uK6MhdTDKoFeCXw/SwmDLsYOKy8i+KJwFOAn1TUbnWmn+Orma/n4xsRmwOXAu/OzP+qrMXqRj/H94lTg+QjYmdgV+DWapqtDvV8fDNzv8ycl5nzgNOBD2emd5N3ofI7SDLzobJX/F1gXeDszPxFRHwAWJSZFwNnAV+IiJsoetaHldv+IiL+Hfgf4CHg+MxcU/V7UHP9HF+A8q+nzYD1ywGaL87M/6n6faixPo/vCcAuwHsj4r3lshdn5h3Vvgs10+fx/XPgXRHxIPAw8JbMvLP6d6Fm+v39rP4407wkSdKQOdO8JEnSkNnhkiRJGjI7XJIkSUNmh0uSJGnI7HBJkiQNmR0uSZKkIbPDJUmSNGR2uCRJkobs/weSpoxJFjNz9wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_feature_importance(clustered_mdi_imp,oob_score=clf.oob_score_, oos_score=oos_score,\n",
    "                        save_fig=True, output_path='img/clustered_mdi.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected the clusters of non-informative features are given the least importnance and the clusters with redundent and informative features are placed above the noise cluster. This is very usefull for detecting features that are non-informative without the presence of some other features within the same cluster. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clustered MDA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an extension to normal MDA to tackle multi-collinearity and (linear or non-linear) substitution effect. Its implementation was also discussed by Dr. Marcos Lopez de Prado in the Clustered Feature Importance [Presentaion Slides](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3517595)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "clustered_mda_imp = mean_decrease_accuracy(clf,X,y,cv_gen=cvGen,clustered_subsets=clusters,random_state=0,scoring=log_loss)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHiCAYAAAA9NBIoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzde7xcVX338c+PqwiEu8pNoqJFQYh6BNSCCAqooNgipSgSi61WeKgX8NLSyqPmqYoKD9aG0qKAWEFUBEHRiEahApqUQLy0XEQhT1DkktBIRAi/54+9j0wmM3PmsmfOzJzP+/WaV2b2XrPXmj1zJnvWXnt9IzORJElS/6w33Q2QJEkadx5wSZIk9ZkHXJIkSX3mAZckSVKfecAlSZLUZx5wSZIk9ZkHXJqRIuLsiPj76W6HJGlm8IBLHYmIX0TE6ohYVXPbocdtHhARy6pqYzsy822Z+aFB1tlMRJwXER+e7nZMJSK2jIj5EfGriHgoIpZGxJsblJtbrnuoLDs/IrasWX9aRDxS8/n5WUT8aYt6N4qIT0TEsrL8HRFxRr9eZ7siYuOI+ExEPFi+zndNUfaMiFgeEQ9ExD9HxIY167eOiEsj4rcR8cuIOKbu+f+rfN0PRsSiiPjjmnVbRsT5EXFPeTuti9fxjxFxZ/m3fWtEnBIRUVfusIj4YdnG+yLi8xGxU836oXyfehERcyJicflZXhwRc1qUvTAi7i7fo1si4i016/aNiAURcX9E/CYiLomI7WvWR0R8tNyv90XEx2r3f0QcHhE/LvfrDyLiOf171eoXD7jUjcMzc7Oa2/LpbExEbDCd9fciItaf7ja0IyI2Ar4N7AK8CNgCOAX4SO2BRkS8G/houW4LYN/yOQvKbUy6ePLzA7wDuDAintyk+vcDE8DewObAy4AbK3x53X6GTgOeSfH6Xga8JyIObVL2fRSvYQ/gWcDzgVNr1n8a+D3wZOANwPyI2L1s2z7AR4AjKfbpucClNZ+dM4AnArMp9tGxjQ6EW7gEOAh4FcX+PRb4K+D/ThaIiCOBfy+XbQvsDjwMXBsRW5XF+v4+tauK74Ty83oZcCGwFXA+cFnd57jWPwKzM3MW8BrgwxHxgnLdVsA5FO/RLsD/AJ+tee5fAUcAewF7AocBby3b8Uzg88DbgC2BrwGXj/L33oyVmd68tX0DfgG8vMm6fYEfACuAm4AData9GfgZxRfNz4G3lss3BVYDjwGrytsOwHnAh2uefwCwrK4d7wVupvji36B83peB3wB3ACe1eB1/2P7ktoH3APcAd1N8+b0KuAW4H/jbmueeBnwJuLh8Pf8J7FWz/tnAwnI//AR4TV2984GvA7+l+KJ9hOI/21XA18py7wNuL7f/U+B1NduYC1wLfBx4oHytr6xZvzXFl/nycv1Xa9YdBiwp2/YDYM823/fjy32zad3yPyvbPau8rQKOqiuzWfncv6jZfxfWlbkHeHGTuq8A3tGibTsDXynf9/uAfyqXr0dxUPPLcvsXAFuU62YDWb6uO4HvT/UZblDv/wMOrnn8IeCiJmUXAa+veXwMcFfN38DvgWfVrP8c8JGaffzDmnWblm3fvnx8L/DCmvV/C1xTt//e16RdBwG/A3auW74PsAbYFYhyH76nrsx6wI+BD7bzPtU9NygOFO8BVlL8He9RrtsE+ERZ50qKz/om5brXUPxNraD4G3t2Vd8JDdp4cPkeR82yO4FD23juH1F8jxzVZP3zgf+pefwD4K/q/t6uL++fCFxZt99XAwe1+1q8DcfNHi5VIiJ2BK4EPkzxH/7JwJcjYruyyD0U/9nPojj4OiMinp+ZvwVeCSzPznvM/hx4NcWvvscofvndBOxI8R/JOyLikDa39RTgCeVz/wH4V+CNwAuA/YB/iIin15R/LUXPwNYUv/y/GhEblqeJvgZ8C3gS8L+Az0fEH9U89xhgHkUvwAUUv14/Vr72w8syt5f1bgH8b4oeoO1rtrEP8N8UvQ0fA86tOQXxOYoej93LNpwBEBHPBz5D8ct5G+BfKH4pb9zG/nkF8I3y/ar15XK/vQh4cXn/K7UFMnMV8I1yG2spT6W8GtiI4sCykeuBd0XE2yPiuXWnWtan+I/+lxQHUTsCF5Wr55a3lwFPpzjw+6e6bb+U4gD5kKk+wxHxvoi4ory/FcV/5jfVbOsmin3eSJS32sc7RcQWFD1eazLzlibb+gawfkTsU77ev6A4aP5V3fZq7+8x+SAzD8vMjzRp1yuAGzLzrtqFmXkDxY+QgygOHp5K8XmvLfMYxfs/+b42fZ8aOBjYv3ztW1IcVN5Xrvs4xd/diyneh/cAj0XEs4AvUPSIbkfxo+VrdT1ObX8nRMQfR8SKFm3cHbg5y6Oc0s00f4+J4lTxQ8B/URxwfb1J0f0pDhxr62r2WWr02VnrPdaImO4jPm+jdaP4FbmK4hfmCsreE4pflp+rK/tN4Lgm2/kq8Dfl/QOo6b0ql53H1D1cf1HzeB/gzrptvB/4bJP6/7D9cturgfXLx5tT9CDsU1N+MXBEef80yl+f5eP1KL5c9ytvvwLWq1n/BeC0mnovaPVam7R3CfDa8v5c4LaadU8s2/sUYHuK/2i2arCN+cCH6pb9N/DSNt73b1P2uDRY9yuK02BvBH7VpMxHgAU1++/35efnIYqelPe0qHt94ATgPyh6LpZPfq4oDvR+A2zQ4HlXA2+vefxHFL2JG/B4D9fTa9a3/Rmm6FVL4Ak1y14B/KLJa/hw2f7tyvfphvL5209+ZurK/yWwsLwfFL1WjwCPsm6P1oUUB7mbU/RI3Q483Obf87/RvFfueuDvgD+uf601Zd4G3DrV+9TgeQdS9B7vy9p/K5O9N3s1eM7fA1+sK/v/KHsh6fE7oUl9F9Ut+zzl3/IUn9c/puhd3bDB+j0pes33q1m2Btit5vEzy30ewG4UveEHUPww+XuKv/H3t/M6vA3PzR4udeOIzNyyvB1RLtsFeH1ErJi8UXzpbA8QEa+MiOvLQaMrKE7XbdtjO2p/le8C7FBX/99SjIlpx32Zuaa8v7r899c161dT9JCsU3cWv/SXUfR47EBxquixmrK/pPiF3ajdDUXEmyJiSc1r2YO199cfejcy86Hy7mYUBwL3Z+YDDTa7C/Duun20c9nmqdxL+V7WtXODsl33lrdtm4wt2b5cP+mL5efnicAzgDdFxFsbVZyZazLz05n5Eoqei3nAZyLi2WX7f5mZjzZ46g4U+37SLykOtmo/E/Wfoaaf4Tqryn9n1SybRXEKuJF5FOOZllCcPvoqxQHUPTx+SrZW7bbeQtGrtTvFf7hvBK6Ixy9WOYni83krxZijL1B8HtvR8H0tTb5n99Y8blZmqvdpLZn5HYrexk8Dv46IcyJiFsVn6QkUB4311no/y7+xu2j+t9XRd0KsfSHQU5n6fWmo3A/XAjsBf11Xx64UPZZ/k5nX1Kyqr2sWsCoL/wUcR7G/7qbYRz+l/fdYQ8IDLlXlLoregS1rbptm5kfKU1ZfpjhV8OTM3JKiq32ymzwbbO+3FD03k57SoEzt8+4C7qirf/PMfFXPr6yxnSfvRMR6FF+uy8vbzuWySU+l+CXeqN3rPI6IXShOaZ4IbFPurx+z9mmFZu4Cto6aqwLr1s2r20dPzMwvtLHdbwOvjIhN65b/KUVvxvXAdeX9P6l7PZtSnDa+utGGM/MXFP8JHd5ofV3Z1Zn5aYqxac8pX9NTmxzkLaf4T3fSUyl6iGoPpOs/Qw0/ww3a8QDFf3571Szei7VPE9W3+8TM3DEzn05x+mxxeZB/C7BBOTi60bb2ohjbd0tmPpaZV5V1v7jc9v2Z+YbMfEpm7k7xvf7DRu1o4NvAPhGxc+3CiNib4jP+HYpe0GXA6+vKrEfx/q/zvjZ4nxrtk7My8wUUB5LPorjQ4l6KMWXPaPCUtd7P8pTlzjT/2+roOyHXvhDoTor9v2fdqdE9afIeN7BB7eso/66/TdHL/Lm6sj+hxWcpM7+UmXtk5jbAByj2w4/abIeGhAdcqsqFwOERcUhErB8RT4hiuoedKH6Vb0xx6ufRiHglxRiOSb8GtinHs0xaArwqisvln0IxbqOVHwIPRsR7I2KTsg17RMQLK3uFa3tBRPxJ+R/9O3j8oOMGioPF95Rjug6gOJC4qOmWitdfOz5sclD0bwDKK87aGq+RmXdTHLz8c0RsVbZh/3L1vwJvK8cCRURsGhGvjojN29j05yj+070kImaX2z0EOIviFMvKzFxJMd7sUxFxaFlmNsXYn2XlNtZRfkYOpcl/ZBHxjvKztElEbBARx1GcPruR4n2/m+JqyU3Lz91Lyqd+AXhnRDwtIjYD/g/F1ZGNesOg9We4kQuAU8v9vBvFacDzmryGHSNih3K/70txWugDAFmMi/sK8MHyNbyEYozg5P76EfDqiHh6+fxXUByg/Ljc9jMiYpuyza+kuBDjwzV1L4wmU0Vk5rcpDpi+HBG7l9vYl+LU2fzMvDUzk2I826kRcUz5PjyF4nTkLB4fI9jqfarfHy8sP4cbUvy9/I5iHNtjFOMMP1nur/Uj4kXlj7YvlvvhoPJ576b4u/tBk/en1++EhRSn+k6KYuqME8vl32nwep4UEUdHxGZlPYdQjCf7Trl+x/L+pzPz7AZ1XUAx/m3HKHou303NZykiXlBudzuKsZdfK3u+NEqm+5ymt9G60foqxX2A71GMT/gNxQDkp5brTqA4sFhB8R/JRaw9RuszFL/6V1CcOngCxVWAD1IMVH0n647henld/TtQ/Cf7K4pf1te3aOt51F2lWLNuA4oDntk1y64F3ljeP421r1K8EXh+Tdndy/2wknWvMPxDvTXLnsnjVw5OjombV+7He4FPltt7S7luLnBt3TYS2LW8vzXFJey/LvfDV2rKHUrxH/gKigOVS4DNy3XfoOZqzAb7bGuKL/tfU5zC+slkm+rKHU9xMLC6LPsv1IwpK/ffIzx+VerdwNnAE5vU+1aKMXQry3b/EDisZv1TKU7R3Vfur7PK5etRXABxF8Xn8cLJdvD4GK4N6upq9Rn+W4oLBybLbkzxuX2wfJ3vqmvTqprn7k/xmX2IosfoDQ327VcpDj7uBI6pWRfAB8vl/0Nxte+xNeuPouj9eaj8HB1St+3bgVe0eF+fQDGVx13le3YbxVWy69WVe2352fltuX++QM3VjVO9T3XbOoji73pV+Z59HtisXLcJcCZFz9VK4Ps8fpXi6yj+plaW79Pu3X4nUIydWzXF993zyte0muJq5OfVrPvD54FibN73ytf9ILAU+Muash8oP2+ram917/HHyv16f3m/9urIa8v3/n6Kv6dNW7Xb23DeonwzJbWp7C3YNTPfON1tkVope+cuycwXTXdbpJnOidMkaUxl5jKKKzklTTPHcEmSJPWZpxQlSZL6zB4uSZKkPvOAS5Ikqc+GetD8tttum7Nnz57uZkiSJE1p8eLF92bmdo3WDfUB1+zZs1m0aNF0N0OSJGlKEfHLZus8pShJktRnU/ZwRcQaillzNwDuoJjheEWL8ldRJMBfm5mH1Sw/CDid4iBvFTA3M2/rrfnVWbhwulsgSeqnAw6Y7hZoJmvnlOLqzJwDEBHnU0S0zGtR/nSK0OG31i2fD7w2M38WEW8HTqWIKJGkgZnzjgOmuwmaLo0i3TUzDEGvSqenFK8DdmxVIDOvpsh8WmcVRdApwBYU2V/riIi/iohFEbHoN7/5TYfNkyRJGj5tD5qPiPUpAkfP7bKutwBfj4jVFOGe+zYqlJnnAOcATExMOCurpEotOXPhdDdB08RTippO7fRwbRIRS4D7KBLtF3RZ1zuBV2XmTsBngU92uR1JkqSR0vYYrojYAriCYgzXWZ1UEhHbAXtl5g3loouBqzpqaZ/5y0eSJPVL22O4MnMlcBJwckRs2GE9DwBbRMSzysevAH7W4TYkSZJGUkcTn2bmjRFxE3A08LlGZSLiGmA3YLOIWAYcn5nfjIi/BL4cEY9RHID9RW9NlyRJGg1THnBl5mZ1jw+fovx+TZZfClzaUeskSZLGgDPNS5Ik9VlXM80DO7PuKcWHKSY7nU8x39YaYF5mXlxu5xpg87Lsk4AfZuYRFbwGSZKkodbVTPOZOQ+YU1+wHBT/psy8NSJ2ABZHxDczc0XtqcaI+DJwWTUvoRpDMAmtJKmPvBpd06mjQfMUM83v2WxlZt5Sc395RNwDbAf8IXsxIjYHDgTe3GHdktQzo31mMKN9Zq4h6FVpewxXzUzzl7dZfm9gI+D2ulWvA67OzAebPM9oH0mSNFba6eGanGl+NrCYNmaaj4jtKcZ4HZeZj9Wt/nPg35o912gfSf1ktM/M5SlFTad2ergmx3DtQtFjdUKrwhExC7gSODUzr69btw2wd7lekiRpRmh7DFdmroyIk4DLImJ+Zj5SXyYiNqKYa+uCzLykwWZeD1yRmb/rusV94i8fSZLULx3Nw5WZNwKTM803chSwPzA3IpaUt9qrGY8GvtBVSyVJkkZUpTPNZ+aFwIUt1h/QSeMkSZLGgTPNS5Ik9Vmn83ABEBHPpcFM85m5T+9NkiRJGi9dHXBl5lIazzS/qv4UZN36q4B9gWsz87Bu6u6XIZgTTZLUR14cpek06FOKp1NkMUqSJM0YXfVwdSszr46IAwZZpyTVMtpnBjPaZ+YagtNYQzdo3mgfSZI0bgbaw9UOo30k9ZPRPjOXY7g0nYauh0uSJGncDF0P13Txl48kSeqXgR5wRcQ1wG7AZhGxDDg+M785yDZIkiQNWqUHXK3m4CrX71dlfZIkSaPAMVySJEl9VvkpRWN/JEmS1tbTAVdErAGWltu5Azi2WexPXXmAOzPzNb3UX6UhmBNNktRHXhyl6dTrKcXVmTknM/cA7gdOaLP8nGE62JIkSeqnKk8pXgfsWeH2JKlyRvvMYEb7zFxDcBqrkkHzEbE+cBBw+RRFn1DG9lwfEUc02ZbRPpIkaaxEZvfpOTVjsmYDi4GDM3NNi/I7ZObyiHg68B3goMy8vVn5iYmJXLRoUdft68QQHPxKkvrIMVzqt4hYnJkTjdZVMoYL2AXYiCnGcGXm8vLfnwMLgef1WL8kSdLQq2QMV2aujIiTgMsiYn5mPlJfJiK2Ah7KzIcjYlvgJcDHqqi/Cv7ykSRJ/VLZxKeZeSNwE3B0kyLPBhZFxE3Ad4GPZOZPq6pfkiRpWPXUw1Uf5ZOZh7co+wPgub3UJ0mSNIqM9pEkSeozo30kSZL6bMoDribxPStaPOV0imkirs3Mw2q2cy4wAQRwCzA3M1d13/RqOS2EJI03L47SdGrnlGKn8T2nA8c2WP7OzNwrM/cE7gRO7KypkiRJo6nTU4pTxvdk5tURcUCD5Q8CREQAmwDdz7gqSV0y2mcGM9pn5hqC01htD5rvIL6n1TY+C/wK2A34VJMyRvtIkqSx0k4P1yYRsYTH43sWdFtZZr65PHD7FPBnwGcblDkHOAeKaJ9u65KkRpacuXC6m6Bp4hguTae2x3DRZnzPVMqsxYuBP+1lO5IkSaOi7TFc7cT3NFOO23pGZt5W3j8c+K/Om9s//vKRJEn90tHEp23E9xAR1wCXAAdFxLKIOIRiKojzI2IpxRQT2wMf7LrVkiRJI2TKHq5O4nvK9fs1WfWSDtolSZI0Noz2kSRJ6rOuon2M75EkSWpft9E+S4E5TcpfBezLutE+5wEvBVaWi+Zm5pKeWl+hIZgTTZLUR14cpek0yGgfgFPKbc0ZpoMtSZKkfhpYtI8kDQOjfWYwo31mriE4jTXQaB9gXkTcHBFnRMTGTeox2keSJI2VQUb7vJ8iR3Ejiuie99JgLi6jfST1k9E+M5djuDSd2jngWp2ZcyJiC+AKijFcZ3VaUWbeXd59uAyxPrnTbfSTf4iSJKlf2j6lmJkrgZOAkyNiw04riojty38DOAL4cafbkCRJGkWDivYB+HxNtM+2wIe7a7IkSdJoGVi0T2Ye2FnTJEmSxoPRPpIkSX1mtI8kSVKfDTLaJyjGbb0eWAPMz8yOr3bslyGYE02S1Edeja7pNMhon7nAzsBumfls4KJOGipJkjSqBhnt89fAMZn5WFnung7rlqSeGe0zgxntM3MNwWmsQUb7PAP4szK25xsR8cwm9RjtI0mSxsogo302Bn6XmRMR8SfAZ4B1ppAw2kdSPxntM3M5hkvTaWDRPsAy4Mvl/UuBz3axjb7xD1GSJPXLwKJ9gK8Ck5OfvhS4pYttSJIkjZxBRvt8BPjTMt7nH4G3dNdkSZKk0TLIaJ8VwKs7ap0kSdIYMNpHkiSpz4z2kSRJ6rOuDrgmo33qYn/uiogty1OH64iIOcB8YBZFtM+8zLy4u2ZXbwjmRJOkSnjVtTR8ujrgqrE6M+cARMT5FFNGzGtS9iHgTZl5a0TsACyOiG82O0CTpFExdLPXD9OM6v6alYBqx3BdB+zYbGVm3pKZt5b3lwP3ANvVl3OmeUmSNG567eEC1or9ObfN8nsDGwG3169zpnlJo2bYZq/3lKI0fHrt4ZqM/bkP2Jo2Yn8iYnuKAfdvngyyliRJGmeVjOFqN/YnImYBVwKnZub1PdZdKX8RSpKkfqlkDFc7sT8RsRFFhuIFmXlJFfVKkiSNgsoGzbcR+3MUsD8wNyKWlLc5VdUvSZI0rHo6pdhJ7E9mXghc2Et9kiRJo8hoH0mSpD6rZFqIWsb+SJIkrW3KA666+J47gGPbiO/ZkLr4nog4CDidoldtFTA3M2+r4kVUwcmQJY0Lr7qWhk87PVxVxffMB16bmT+LiLcDpwJze34FkjTNjPZpwV+zEtD5GK5e4nuSIrgaYAtgeaNtGO0jSZLGTdtjuCqI73kL8PWIWA08COzb6HlG+0gaNUb7SJpKOz1cVcX3vBN4VWbuBHwW+GR3TZYkSRotbY/h6iW+JyK2A/bKzBvKYhcDV/XU8or5i1CSJPVL22O4eozveQDYIiKeVT5+BfCz7posSZI0Wjqahyszb4yIyfie+rm24PH4nm0iYm65bG5mLomIvwS+HBGPURyA/UX3zZYkSRodkTm849InJiZy0aJF090MSZKkKUXE4sycaLTOaB9JkqQ+6yrapya+Zy/gd+Xi3wFPazYLfc1zZ1GM37o0M0/spv5+cG4+SePCi4Ck4dNVD1dmLi1nn/9tZm6SmZsAl1NcwTiVDwHf66ZeSZKkUVRlePV1wJ6tCkTEC4AnU0wJ0fAcpySNGqN9WvD0gQRUNIarZhb6y1uUWQ/4BHDKFNsy2keSJI2VXnu4Jmehnw0spvUs9G8Hvp6Zd0VE00JG+0gaNUb7SJpKrz1cq8uxXLtQ5Ca2GsP1IuDEiPgF8HHgTRHxkR7rlyRJGnqVjOHKzJURcRJwWUTMz8xHGpR5w+T9clLUicx8XxX1V8FfhJIkqV8qm4crM28EJmehlyRJUqmnHq7M3Kzu8eFtPu884Lxe6pYkSRoVzjQvSZLUZ1XOwwWsNQt9rYczc5+q65IkSRoFPR1wRcQaYGm5nTuAYzNzKTCnQdldgK8A6wMbAp/KzLN7qb9Kzs0naVx4EZA0fCqZFiIz9wDup/W0EHcDLy6nkdgHeF9E7NBj/ZIkSUNvYNE+mfn7mocb4/gxSWPCaJ8WPH0gAQOM9inL7RwRNwN3AR/NzOUNyhjtI0mSxkpkdp+eUzOGazZFtM/BmbmmjeftAHwVODwzf92s3MTERC5atKjr9nXCH2GSxoVjuKTpERGLM3Oi0bpBRvv8Qdmz9RNgvx7rlyRJGnoDi/aJiJ2A+zJzdURsBbwE+GQV9VfBX4SSJKlfBhnt82zghoi4Cfge8PFyCglJkqSxNrBon8xcQIurGCVJksaVUzNIkiT1mdE+kiRJfTblAVeT+J4VTcrOAeZTRPesAeZl5sXlugOBj1NczbgYOD4zH63iRVTBaSEkjQsvApKGTzunFDuJ73kIeFNm7g4cCpwZEVtGxHrA+cDR5XZ+CRzXY9slSZJGQqenFKeK77ml5v7yiLgH2I6ix+vhmvULgPcD53ZYvyQNHaN9WvD0gQR0MGi+3fiemvJ7U5w+vB24F9gwIiZnXz0S2LnJ84z2kSRJY6WdHq5NImIJj8f3LJjqCRGxPcXA+eMy87Fy2dHAGRGxMfAtoOH4rcw8BzgHimifNtonSdNqyZkLp7sJa3EMlzR82h7DRZvxPRExC7gSODUzr59cnpnXZeZ+mbk38H3g1u6bLUmSNDraHsPVZnzPRsClwAWZeUnduidl5j1lD9d7gXk9tr1S/iKUJEn90tHEp23E9xwF7A/MjYgl5W1Oue6UiPgZcDPwtcz8TreNliRJGiWRObzDpCYmJnLRokXT3QxJkqQpRcTizJxotM5oH0mSpD7rKtrH+B5JkqT2dRvtsxSY06DsZLTPLNaN9nkacBGwNfCf5XZ+X9Hr6Jlz80kaF14EJA2fgUT7lOs+CpyRmc8EHgCO76HdkiRJI2Mg0T4RsRI4EDimXH0+cBpFb5gkjTSjfVrw9IEEDC7aZxtgRWZOzi6/DNixyfOM9pEkSWNlINE+ERENijWcj8JoH0mjxmgfSVMZVLTPvcCWETF5gLcTsLy7JkuSJI2WgUT7ZGZGxHeBIymuVDwOuKzn1lfIX4SSJKlfBhnt817gXRFxG8WYrnO7bLMkSdJImbKHKzM3q3t8eIuyFwIXNln3c2DvThsoSZI06oz2kSRJ6jOjfSRJkvqsqwOuFtE+q+pPQdas2wX4CrA+sCHwqcw8u5v6+8G5+SSNCy8CkoZPVwdcXbobeHFmPhwRmwE/jojLM9PpISRJ0lgb2AFXXVD1xjh+TNKYMNqnBU8fSMCAD3oiYueIuBm4C/hoo94to30kSdK4iczq0nNajeGqK7cD8FXg8Mz8dbNyExMTuWjRosra14o/wiSNC8dwSdMjIhZn5kSjdYMcw/UHmbk8In4C7Ad8aTraUM8vKEmS1C8DO6UYETtFxCbl/a2AlwD/Paj6JUmSpssge7ieDXwiIhII4OPl9BKSJEljrdIDrlbjtzJzAbBnlfVJkiSNAqdmkCRJ6rPKTyka+yNJkrS2KQ+4ImINsLQsewdwbGauaFJ2DjCfIrpnDZAgiqAAACAASURBVDAvMy8u130emAAeAX4IvDUzH6niRVTBaSEkjQuvupaGTzunFFdn5pzM3AO4HzihRdmHgDdl5u7AocCZETE55/Hngd2A5wKbAG/pvtmSJEmjo9NTitfRYuB7Zt5Sc395RNwDbAesyMyvT66LiB8CO3VYtyQNJaN9WvD0gQR0MGg+ItYHDgIub7P83sBGwO11yzcEjgWuavI8o30kSdJYmTLap2YM12xgMXBwZq6Z4jnbAwuB4zLz+rp1/wr8NjPfMVXjjPaRpM45hkuaHr1G+6zOzDkRsQVwBcUYrrNaVDYLuBI4tcHB1gcoTjG+td3GD4pfUJIkqV/aPqWYmSuBk4CTy9OC64iIjYBLgQsy85K6dW8BDgH+PDMf677JkiRJo6WjiU8z80bgJuDoJkWOAvYH5kbEkvI2p1x3NvBk4Lpy+T9022hJkqRRMuUpxfq4nsw8vEXZC4ELm6wbZG6jJEnS0DDaR5Ikqc+66nUyvkeSJKl93Ub7LAXmNCg7Ge0zi3Wjfc4DXgqsLIvPzcwlFbyGSjgthKRx4VXX0vBpe1oIgIg4n2JaiHlNyk5G+9waETsAiyPimzXZi6dk5pd6brUkDRFnmm/BX7MS0PkYruuAHZutzMxbMvPW8v5yYDLap23ONC9JksZN22O4aqJ9zm2zfKNon3nldBBXA+/LzIfrn5eZ5wDnQDHTfLvtk6TpsuTMhdPdhLV4SlEaPu30cG0SEUuA+4CtgQVTPaGM9vkc8OaaSU7fD+wGvLDcznu7arEkSdKIGVi0T2beXd59OCI+C5zcfbOr5y9CSZLUL4OM9tm+/DeAI4Afd9toSZKkUTLIaJ/PR8RSiikmtgU+3GWbJUmSRsogo30O7Lh1kiRJY8BoH0mSpD7r6oArIp5bni7MiFhd3h6IiJbT7UXEUyPiWxHxs4j4aUTM7qZ+SZKkUdJVluJktE9ErJo85djGLPQAF1DE/SyIiM2Ax1qUHSgnQ5Y0LrzqWho+XR1wNXEdsGezlRHxHGCDzFwAkJmrKqxbkqaN0T4t+GtWAioaw1UzC/3lLYo9C1gREV+JiBsj4vTyefXbMtpHkiSNlV57uCZnoZ8NLKb1LPQbAPsBzwPuBC4G5lIXFWS0j6RRY7SPpKn02sO1OjPnALtQ5Cae0KLsMuDGzPx5Zj4KfBV4fo/1S5IkDb1KxnBl5sqIOAm4LCLmZ+YjDYr9CNgqIrbLzN8ABwKLqqi/Cv4ilCRJ/VLZPFxTzUKfmWso8hOvLmecD+Bfq6pfkiRpWPXUw9XJLPTl+gW0uJJRkiRpHDnTvCRJUp9VOQ8XUMxCD3yubvHDmblP1XVJkiSNgp4OuCJiDbC03M4dwLGTs9A3KPsy4IyaRbsBR2fmV3tpQ1Wcm0/SsPKiHmn0VTItRGbuAdxPi2khMvO7Zdk5FFcoPgR8q8f6JUmSht7Aon3qHAl8IzMfqrB+SX0wdLE1M9EwRfXMRJ4CUQUGGe1T62jgC022ZbSPJEkaK5HZfXpOzRiu2RTRPgeX8221es72wM3ADk0mSP2DiYmJXLRoMHOj+gNG0rByDJc0GiJicWZONFo3yGifSUcBl051sCVJkjQuBhntM+nPgfdXUW+V/AUpSZL6ZWDRPgARMRvYGfheVfVKkiQNu0FH+/wC2LGXOiVJkkaN0T6SJEl9ZrSPJElSn015wNUkvmdFi6ecTjFNxLWZeVjNdj4PTACPAD8E3jpMVyo6LYSkYeVFPdLoa+eUYtvxPaXTgWMbLP88RX7ic4FNgLd00lBJkqRR1ekpxSnjezLz6og4oMHyr0/ej4gfAjt1WLekaWC0zxAw2md6eQpEFWh70HwX8T3NtrMhRQ/YVU3WG+0jSZLGSjs9XJtExBIej+9Z0GOd/wx8PzOvabQyM88BzoEi2qfHuiT1aMmZC6e7CTOeY7ik0df2GC46i+9pKCI+AGwHvKvbbUiSJI2atsdwdRjfs46IeAtwCHBQZj7WYTv7zl+QkiSpXzqa+LTN+J5rgEuAgyJiWUQcUq46G3gycF1ELImIf+iyzZIkSSNlyh6uLuJ79muyvPJJViVJkkaB0T6SJEl91lWvk/E9kiRJ7evqgCszlwJz6pdHxKr6U5A16+YA84FZwBpgXmZe3E39/eC8dpKGlRf1SKNvkOOqHgLelJm3RsQOwOKI+OYUuYySJEkjb2AHXJl5S8395RFxD8WcXB5wSUPMaJ8hYLTP9PIUiCowLYPmI2JviklUb2+wzmgfSZI0VgY+VUNEbE8x4P64RhOgGu0jDRejfaafY7ik0TfQHq6ImAVcCZyamdcPsm5JkqTpMrAerojYCLgUuCAzLxlUve3yF6QkSeqXQfZwHQXsD8wto32WlFNFSJIkjbVKe7iazcFVrrsQuLDK+iRJkkaB0T6SJEl9VvkYLmN/JEmS1jblAVdErAGWlmXvAI5tNjt8TXzPhjSJ74mITwFvbnX6cTo4r52kYeVFPdLoa+eU4urMnJOZewD3Aye0KDsZ37M7cChwZkT8YY7kiJjAOZMlSdIM0+kpxeuAPZutbBXfExHrA6cDxwCv66KtkqaB0T5DwJ+p08tTIKpA24PmywOmg4DL2yxfH99zInB5Zt49xfOM9pEkSWOlnR6uTSJiCTAbWAwsmOoJ9fE9EbED8HrggKmea7SPNFyM9pl+juGSRl/bY7iAXSh6rFqN4WoW3/M8YFfgtoj4BfDEiLit61ZLkiSNkLbHcGXmyog4CbgsIuZn5iP1ZZrF92TmlcBTasqtysxde2t6tfwFKUmS+qWjiU8z80bgJuDoJkWM75EkSaozZQ9X/XxZmXl4i7JtxfcM2xxckiRJ/WS0jyRJUp91Fe1jfI8kSVL7ujrgysylwJy62J+7ImLLZrE/ABFxFbAvcG1mHtZN3f3ivHaSposX7Ujjr9dTip3E/kAx0/yxPdYpSZI0Urrq4WqiZewPQGZeHREHVFinpCFmLFCbjO5pn6cjNKIqGTTfaezPFNsy2keSJI2VXnu4Oo79mYrRPtL4MBaoPY7hksZfrwdcqzNzTkRsAVxBMYbrrN6bNXh+4UmSpH6p5JRiZq4ETgJOjogNq9imJEnSuKhs4tM2Yn+IiGuAS4CDImJZRBxSVf2SJEnDqqdTip3E/pTr9+ulPkmSpFFktI8kSVKfVTkPF2DsjyRJUr2eDrjqon3uAI6djP1pUv6jwKvLhx/KzIt7qb9KzqUnabp4lbQ0/gYW7RMRrwaeT3Ewtg9wSkTM6rF+SZKkoTfIaJ/nAN/LzEeBRyPiJuBQ4IsVtkHSEDHap01G+7TP0xEaUYOM9rkJeGVEPDEitgVeBuzcYFtG+0iSpLEysGifzPxWRLwQ+AHwG4oesUcblDPaRxoTRvu0xzFc0vgbaLRPZs4D5gFExL8Dt/ZYf2X8wpMkSf0ysGifiFg/IrYp7+9JMd7rW1XUL0mSNMwqGzSfmTeWA+GPZt15uAA2BK6JCIAHgTeWA+glSZLG2sCifTLzdxRXKkqSJM0oRvtIkiT1mdE+kiRJfVbpAVdErCpPMzaL9jkOOLV8+OHMPL/K+nvhXHqSpotXSUvjr/IermYiYmvgA8AEkMDiiLg8Mx8YVBskDZYzzbfJmebb569jjahBjuE6BFiQmfeXB1kLKKJ91uJM85IkadwMrIcL2BG4q+bxsnLZWpxpXhofzjTfHk8pSuNvkD1c0WCZB1SSJGnsDbKHaxlwQM3jnYCFA6y/JX9hSpKkfhlkD9c3gYMjYquI2Ao4uFwmSZI01gbWw5WZ90fEh4AflYs+mJn3D6p+SZKk6VLpAVd91E+D9Z8BPlNlnZIkScPOaB9JkqQ+m/KAKyLWRMSSiPhxRHwtIlpO0RcR15bPWVk+b0lE3BARTyv/vTUiLo6Ijap7GZIkScOrnVOKqzNzDkBEnA+cAMxrUf4DwBOBt2bmYZMLI+KLwBmZeVFEnA0cD8zvuuUVc/JiSdPFq6Sl8dfpGK7rgD1bFcjMqyPigNplERHAgcAx5aLzgdMYogMuSdUz2qdNRvu0z1/HGlFtj+GKiPWBg4DLu6hnG2BFZj5aPm44y3xZj9E+kiRprLTTw7VJRCwBZgOLKTIQO9X2LPNG+0jjw2if9nhKURp/7fRwTY7h2gXYiGIMV6fuBbaMiMkDvJ2A5V1sR5IkaeS0PYYrM1dGxEnAZRExPzMf6eC5GRHfBY4ELgKOAy7ruLV95C9MSZLULx3Nw5WZNwI3AUc3KxMR1wCXAAdFxLKIOKRc9V7gXRFxG8WYrnO7a7IkSdJombKHq372+Mw8fIry+zVZ/nNg745aJ0mSNAacaV6SJKnPuspSjIjnAp+rW/xwZu7Te5MkSZLGy5QHXBGxBlhalr0DODYzlwJzGpSdQzGZ6SxgDTAvMy8u1wXwYeD15br5mXlWRa+jZ86lJ2m6eNGONP7anhYiM/cA7qf1tBAPAW/KzN2BQ4Eza7IX5wI7A7tl5rMprlaUJEkae5VG+2TmLTX3l0fEPcB2wArgr4FjMvOxcv09nTdX0igx2qdNRvu0z9MRGlF9i/aJiL0pJkq9vVz0DODPytieb0TEM5s8z2gfSZI0VvoS7RMR21MMqj9uskcL2Bj4XWZORMSfAJ8B1plCwmgfaXwY7dMex3BJ46/yaJ+ImAVcCZyamdfXrFoGfLm8fyktTk1KkiSNk0qjfSJiI4qDqQsy85K61V8FDqTo2XopcAtDxF+YkiSpX6qO9jkK2B+YGxFLytvk9BEfAf40IpYC/wi8pcs2S5IkjZRKo30y80LgwibrVgCv7rSBkiRJo85oH0mSpD4z2keSJKnPKo32KctfBewLXJuZhzVY/yngzfWnKqebc+lJmi5etCONv6qjfQBOB45ttCIiJnBOZUmSNMNUGu0DkJlXR8QB9cvLmepPB44BXtdhvZJGkNE+bfJnaPs8HaER1bdonwZOBC7PzLunqMdoH0mSNFb6Eu1TLyJ2AF4PHDBVWaN9pPFhtE97HMMljb/Ko32aeB6wK3BbRPwCeGJE3NbFdiRJkkZOpdE+LZ57JfCUyccRsSozd+2sqf3lL0xJktQvVUf7EBHXAJcAB0XEsog4pLcmSpIkjbZKo33K9ft1uk1JkqRxZrSPJElSnxntI0mS1GcDi/Ypx3ZtXj58EvDDzDyit+ZXx7n0JE0XL9qRxt/Aon0yc79yO3MoZqz/SsetlSRJGkEDi/aZFBGbAwcCb+6wbkkjxmifNhnt0z5PR2hEDTLaZ9LrgKsz88Em9RjtI0mSxspAon3q/Dnwb81WGu0jjQ+jfdrjGC5p/A0q2geAiNgG2Bu4stttSJIkjZqBRPvUeD1wRWb+rovn9pW/MCVJUr8MOtrnaOAL3TRUkiRpVA002iczD2i7ZZIkSWPCaB9JkqQ+M9pHkiSpzyqN9omIOcB8YBawBpiXmReX684FJoAAbgHmZuaqil5Hz5xLT9J08aIdafxVHe3zEPCmzNwdOBQ4MyIm51B+Z2bulZl7AncCJ/bScEmSpFFRabRPZt5Sc395RNwDbAesmJxZPiIC2ARwUlNpzBnt0yajfdrn6QiNqL5F+0TE3hQTpd5es+yzwK+A3YBPNXme0T6SJGmsRGbrjqaaMVyzKaJ9Ds7MNVM8Z3tgIXBcZl5ft259ioOtH2XmZ1ttZ2JiIhctWjTFS6iGP5okTRfHcEnjISIWZ+ZEo3XtnFJcnZlzImIL4AqKMVxntahsFkV0z6n1B1sAmbkmIi4GTgFaHnANkl94kiSpX9o+pZiZK4GTgJMjYsNGZSJiI+BS4ILMvKRmeUTErpP3gcOB/+ql4ZIkSaOi6mifo4D9gbkRsaS8zaGYCuL8iFhKcXpye+CD3TdbkiRpdFQa7ZOZFwIXNln9ks6aJkmSNB6M9pEkSeozo30kSZL6bJDRPicC7wCeAWyXmfdW9SKq4LQQkqaLV0lL42+Q0T7/Abwc+GUvDZYkSRo1g4z2uRGgmBVC0kxgtE+bjPZpn6cjNKIGGu3T5vOM9pEkSWOlnR6uTSJiCY9H+yyY6glltM/nKKJ9HuukQZl5DnAOFNE+nTxX0nBZcubC6W7CSHAMlzT+Bh7tM6z8wpMkSf0ykGgfSZKkmWxQ0T5ExEkRsQzYCbg5Iv6th3ZLkiSNjMgc3mFSExMTuWjRouluhiRJ0pQiYnFmTjRaZ7SPJElSnxntI0mS1GddHXC1iPZZlZmbtXpueRXjz4BLM/PEburvB+fSk4abVxJLGmXTcUrxQ8D3pqFeSZKkadFVD1e3IuIFwJOBq4CGg8qkmcTomw4Yf9M+u+yloTOwHq6IWA/4BHDKFOWM9pEkSWNlkD1cbwe+npl3tQqwNtpHM4nRN+1zDJekUTbIA64XAftFxNuBzYCNykH27xtgG5ryy1ySJPXLwA64MvMNk/cjYi4wMSwHW5IkSf3kxKeSJEl9VmkP11RzcNWUOw84r8q6JUmShpU9XJIkSX1W+RguY38kSZLW1tMBV0SsAZaW27kDOLZZ7E9Z/mPAqyl61hYAf5OZQzH1g/MESsPNK4kljbJee7hWZ+YcgIg4HzgBmNeoYES8GHgJsGe56FrgpcDCHtsgjSxnmu+AM823z1+Q0tCpcgzXdcCOLdYn8ARgI2BjYEPg1/WFnGlekiSNm0rGcEXE+sBBwLnNymTmdRHxXeBuIIB/ysyfNSjnTPOaMZxpvn2eUpQ0ynrt4dokIpYA9wFbU4zLaigidgWeDexE0RN2YETs32P9kiRJQ6+SMVwRsQVwBcUYrrOalH0dcH1mrgKIiG8A+wLf77ENlfDXsyRJ6pdKxnBl5krgJODkiNiwSbE7gZdGxAZlmZcC65xSlCRJGjeVDZrPzBuBm4CjmxT5EnA7xTQSNwE3ZebXqqpfkiRpWPV0SrE+yiczD29Rdg3w1l7qkyRJGkVG+0iSJPVZTwdcEbEmIpZExI8j4msRsWVEPLdcVnu7oSx/XETcWt6Oq+YlSJIkDbfKZ5rPzHk0iPaJiK2BDwATFJOgLo6IyzPzgR7bUAknZpaGm1cSSxplVYZXX8fjsT2NHAIsyMz7ASJiAXAo8IUK2yCNFKN9OmC0T/v8BSkNnUrGcNXMNH95i2I7AnfVPF5Ggyggo30kSdK46bWHa3Km+dnAYlrMNE8R51Nvnegeo300kxjt0z5PKUoaZb32cE2O4dqFIpT6hBZllwE71zzeCVjeY/2SJElDr5IxXJm5MiJOAi6LiPmZ+UiDYt8E/k9EbFU+Phh4fxX1V8Ffz5IkqV8GNtN8OVj+Q8CPytsHJwfQS5IkjbOBzTRfrv8M8Jle6pQkSRo1zjQvSZLUZ1XOwwVARDwX+Fzd4oczc5+q65IkSRoFUx5wRcQaYGlZ9g7g2Mxc0eIpp1NME3FtZh5Ws50DgY9TXM24GDg+Mx/tvunVcp5Aabh5YYukUdbOKcXVmTknM/cA7qf11A9QHHAdW7sgItYDzgeOLrfzS8AsRUmSNCN0ekpxqvgeMvPqiDigbvE2FKcVbykfL6CYEuLcDuuXxorRPh0w2qd9dtlLQ6ftQfNtxvc0cy+wYURMlI+PZO1JUGvrMdpHkiSNlXZ6uDqJ72koMzMijgbOiIiNgW8BDcdvGe2jmcRon/Y5hkvSKGt7DBftxfc0lZnXZeZ+mbk38H3g1m62I0mSNGraHsPVZnxPUxHxpMy8p+zhei8wr8O29pW/niVJUr90NPHpVPE9ABFxDXAJcFBELIuIQ8pVp0TEz4Cbga9l5ne6bLMkSdJImbKHq4v4nv2aLD8FOKWj1kmSJI0Bo30kSZL6rKtoH+N7JEmS2tdttM9SYE6T8lcB+7JutM+JwDuAZwDbZea9vTe/Os4TKA03L2yRNMoGEu1T+g/g5RSxPpIkSTPGoKJ9Jq9wJCI6rFIaX0b7dMBon/bZZS8NnUFF+7TNaB9JkjRuBhLt0wmjfTSTGO3TPsdwSRplA4v2kSRJmqkGFu0z7Pz1LEmS+mVg0T4RcVJELAN2Am6OiH/rvtmSJEmjY5DRPmcBZ3XUOkmSpDFgtI8kSVKfGe0jSZLUZ131cJXRPnsBV5ez0M8BLomI05o9JyLeFRE/jYibI+LqiNiluyb3h/MESpKkfunllOLDwJ9ExLZtlr8RmMjMPYEvAR/roW5JkqSR0dUpxdKjFBOUvhP4u6kKZ+Z3ax5eD7yxh7qrU84HMWcFa0eH2OUlSZIq0uug+U8Db4iILTp83vHANxqtMNpHkiSNm156uMjMByPiAuAkYHU7z4mINwITwEubbHOw0T5lT9aShU5+KkmS+qOKaSHOpOix2nSqghHxcorTj6/JzIcrqFuSJGno9XzAlZn3A1+kOOhqKiKeB/wLxcHWPb3WWzV7tyRJUr9UNfHpJ4CprlY8HdiMYvqIJRFxeUV1S5IkDbWux3DVRv5k5q+BJ05R/uXd1iVJkjTKjPaRJEnqs56uUmwkIv4OeH3d4ksyc17VdUmSJI2Crnu4IiIj4hM1j0+OiNMyc95k3E/NbV5E7B8R/xkRj0bEkdU0vzrOcypJkvplkNE+dwJzgX/voU5JkqSRM8hon18ARMRjPdRZPaN9JElSn01XtE9TRvtIkqRxM/Bonza2abSPJEkaKwON9hlmHmxJkqR+GVi0jyRJ0kw1sGifiHhhRCyjmKPrXyLiJxXVLUmSNNQGGe3zI2CnbuuTJEkaVUb7SJIk9ZnRPpIkSX3W9QFXRCTwycx8d/n4ZGCzzDwNWOfgKiI2Bi4AXgDcB/zZ5GSow2DhQq9UlCRJ/THIaJ/jgQcyc1fgDOCjPdQtSZI0MgYW7QO8FjitvP8l4J8iIjKz/5ObtmK0jyRJ6rNBRvvsCNwFkJmPAiuBbeoLGe0jSZLGzSCjfaLRJhps02gfSZI0VgYZ7bMM2BkgIjYAtgDur6D+SniwJUmS+mWQ0T6XA8eV948EvjPt47ckSZIGYGDRPsC5wDYRcRvwLuB9FdUtSZI01AYZ7fM71p0QVZIkaewZ7SNJktRnRvtIkiT1Wdc9XBGREfGJmscnR8RpmTkvM+fU3ebVlDuyfO5Er42vkvOcSpKkfumlh2sy2ucfM/Pedp4QEZtTzNl1Qw/1VsuZ5iVJUp/1MoarNtqnXR8CPgb8rlkBZ5qXJEnjptcxXJ8Gbo6Ij01VMCKeB+ycmVdExMnNyjnTvCRJGjcDifaJiPWAM4C5vdQnSZI0igYV7bM5sAewMCJ+AewLXD5MA+ft3ZIkSf0ykGifzFyZmdtm5uzMnA1cD7wmMxf1Wr8kSdKwG2S0jyRJ0ow0sGifuuce0G29kiRJo8ZoH0mSpD4z2keSJKnPuj7giogEPpmZ7y4fnwxslpmnAescXEXE24ATgDXAKuCvMvOn3dZftYULvVJRkiT1xyCjff49M88GiIjXAJ8EDu2h/moY7SNJkvpsYNE+mflgzcNNgYazyBvtI0mSxs3Aon0AIuIE4F3ARsCBjcoY7SNJksZNT1cplr1Wk9E+7ZT/dGY+A3gvcGovdUuSJI2KQUX71LsIOKKCuitj75YkSeqXgUT7AETEM2sevhq4tde6JUmSRkFV83B9AjhxijInRsTLgUeAB4DjKqpbkiRpqEVm/8eldysifgP8ckDVbQu0M72F2uc+rZ77tHru0+q5T6vnPq1eP/bpLpm5XaMVQ33ANUgRsSgzJ6a7HePEfVo992n13KfVc59Wz31avUHvU6N9JEmS+qzyA67ywMqDK0mSpFIV00KMi3OmuwFjyH1aPfdp9dyn1XOfVs99Wr2B7lPHcEmSJPWZPVySJEl9NiMOuCLi0Ij474i4LSLe12D9xhFxcbn+hoiYXbPu/eXy/46IQwbZ7mHW7T6NiNkRsToilpS3swfd9mHVxj7dPyL+MyIejYgj69YdFxG3ljfnuCv1uE/X1HxOLx9cq4dbG/v0XRHx04i4OSKujohdatb5OW2gx33q57SBNvbp2yJiabnfro2I59Ss68//+5k51jdgfeB24OkUodk3Ac+pK/N24Ozy/tHAxeX955TlNwaeVm5n/el+TdN963GfzgZ+PN2vYdhube7T2cCeFPmlR9Ys3xr4efnvVuX9rab7NU33rZd9Wq5bNd2vYdhube7TlwFPLO//dc3fvp/Tivdp+djPaXf7dFbN/dcAV5X3+/b//kzo4dobuC0zf56Zv6fIcXxtXZnXAueX978EHBQRUS6/KDMfzsw7gNvK7c10vexTNTblPs3MX2TmzcBjdc89BFiQmfdn5gPAAuDQQTR6yPWyT9VYO/v0u5n5UPnwemCn8r6f08Z62adqrJ19+mDNw02ByQHtfft/fyYccO0I3FXzeFm5rGGZzHwUWAls0+ZzZ6Je9inA0yLixoj4XkTs1+/GjohePmt+Thvrdb88ISIWRcT1EXFEtU0bWZ3u0+OBb3T53Jmil30Kfk4baWufRsQJEXE78DHgpE6e243K5+EaQo16Veovzfz/7d09axRRGMXx/0FROwlop4EE1iZVMDZa2EhARRstRGzUj7FWfoMUKfwGgo1sJ0i0jiiBJYKvlaworJ2F1bW4VxjCLs7O7M3MMucHl329cOfwwDwzO5NM+06ZuV1UJ9PvwHIIYSzpPPBc0tqBo40uqlNrrtPJ6uayHEIYSVoFdiQNQwhf5rS2RVU6U0n3gA3g8qxzO6ZOpuA6naRUpiGEbWBb0l3gEfF/PGer0y6c4foGnC28PgOMpn1H0lHgJPCr5NwuqpxpOk07BgghvCX+Pn4u+4rbr06tuU4nq5VLCGGUHr8Cr4H1eS5uQZXKVNIVoA/cDCH8mWVuB9XJ1HU62ay19hT4d3YwW512oeF6A/QkrUg6RryA++CdHANiZwtwG9gJ8eq5AXAn3XG3AvSA3UNad5tVzlTSaUlHANIRWY948WzXlcl0mhfApqQlSUvAZnqv6ypnmrI8np6fAi4B77OtdHH85gHw3wAAAO5JREFUN1NJ68ATYmPws/CR63Syypm6Tqcqk2mv8PI68Ck9z7ffb/pugsMYwDXgI/FsSj+995hYvAAngGfEi+N2gdXC3H6a9wG42vS2tGVUzRS4BewT7wJ5B9xoelvaMkpkeoF49PUbGAP7hbkPUtafgftNb0tbRtVMgYvAMNXpEHjY9La0ZZTI9CXwA9hLY1CY6zqdY6au01qZbqV90R7wClgrzM2y3/dfmjczMzPLrAs/KZqZmZk1yg2XmZmZWWZuuMzMzMwyc8NlZmZmlpkbLjMzM7PM3HCZmZmZZeaGy8zMzCwzN1xmZmZmmf0FT3yApL8o8WoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_feature_importance(clustered_mda_imp,oob_score=clf.oob_score_, oos_score=oos_score,\n",
    "                        save_fig=True, output_path='img/clustered_mda.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The clustered MDA has also correctly identified the noisy cluster and placed it below. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Caveat\n",
    "Now that we saw how to implement the CFI with MDI and MDA, we have to discuss the *caveat* of normal ONC algorithm that was mentioned in the Step -1 of this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand the caveat of the normal ONC algorithm, we need a understanding of how it works. ONC finds the optimal number of clusters as well as the composition of those clusters, where each feature belongs to one and only one cluster. Features that\n",
    "belong to the same cluster share a large amount of information, and features that belong to different clusters share only a relatively small amount of information. \n",
    "<br>The consistency composition of the clusters are determined by the [silhouette score](https://en.wikipedia.org/wiki/Silhouette_(clustering)) of the features. The silhouette ranges from −1 to +1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. So, there may be some features with low silhouette score and this is a problem, because ONC cannot assign one feature to multiple clusters. \n",
    "<br>In this case, the following transformation may help reduce the multicollinearity of the system :"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each cluster $k = 1 . . . K$, replace the features included in that cluster with residual features, so that it do not contain any information outside cluster $k$. That is let $D_{k}$ be the subset of index features $D = {1,...,F}$ included in cluster $k$, where $D_{k}\\subset{D}\\ , ||D_{k}|| > 0 \\ , \\forall{k}\\ ; \\ D_{k} \\bigcap D_{l} = \\Phi\\ , \\forall k \\ne l\\ ; \\bigcup \\limits _{k=1} ^{k} D_{k} = D$ . Then, for a given feature $X_{i}$ where $i \\in D_{k}$, we compute the residual feature $\\hat \\varepsilon _{i}$ by fitting the following equation for regression -\n",
    "\n",
    "$$X_{n,j} = \\alpha _{i} + \\sum \\limits _{j  \\in \\{ \\bigcup _{l<k}\\ D_{l} \\} } \\beta _{i,j} X_{n,j} + \\varepsilon _{n,i}$$\n",
    "\n",
    "Where $n = 1,....,N$ is the index of observations per feature. But if the degrees of freedom in the above regression is too low, one option is to use as regressors linear combinations of the features within each cluster by following a minimum variance weighting scheme so that only $K-1$ betas need to be estimated.\n",
    "This transformation is not necessary if the silhouette scores clearly indicate that features belong to their respective clusters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlfinlab.clustering.onc import get_onc_clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "corr0,clstrs,silh = get_onc_clusters(X.corr())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2271a1229e8>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2IAAAIbCAYAAACJ9S0vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdf5Rf9X3f+ecLRXKVoxKBQ2oMjoxjZ+01IuNWRWx8cFKrCbRZOW6TgNyWH25ctV17dU5ySNOc5SR7nPrUKWntJtu6VWwSQGnixSmO+BGoliYN3YrUQzVCYNeATRxrhcsWGWEfiGxm3vvHfOWdGWbm++t+Z773q+eDcw/fe+/n3s97vvOVNO/53M/7k6pCkiRJkrR2zlnvACRJkiTpbGMiJkmSJElrzERMkiRJktaYiZgkSZIkrTETMUmSJElaYyZikiRJkrTGTMQkSZIkTbwktyZ5NsljK5xPkl9J8lSSR5P8+QXnbkjyZGe7oYl4uiZiSWaTzCR5LMndSbau0nYqyeEkj3eCv3bBuXcm+S+d+9yW5Nua+AIkSZIkqQe/AVy9yvm/Aryps+0FPgaQ5HzgF4CdwOXALyQ5b9hgehkRe6mqpqrqUuAk8P5V2r4IXF9Vb2X+i/xokq1JzgFuA/Z07vMloJFMUpIkSZK6qao/ZD6fWcmPArfXvIeBrUkuBK4CDlXVyar6KnCI1RO6nvT7aOJh4KKVTlbVE1X1ZOf1CeBZ4ALg1cDpqnqi0/QQ8GP9hytJkiRJI3ER8OUF+8c7x1Y6PpSeHw9MsgHYBXyix/aXA5uALwAFbEyyo6qmgR8HXrfCdXuZHwrkY//ku/7C3uu+o9cQl/XNenmo63X22pBmplD+lde+rZH7TKJv/Mjl6x2C1tjsxqx3CGeHcXqba70DWGzLU8+vdwgT76Vtw/3sdsbLr2rmg5yGPoOZa+Y+1VCFhv/4OzeN05/0nsx95XtH+jfChguf/Lt08oiO/VW1v49bLPee1irHh9JLIrY5yQzweuAR5kezVtUZwrsDuKGq5jrH9gAfSfIq4N8By2ZInTdrP4z+myVJkiRpMizMIwZ0nMWDRRcDJzrHf3DJ8T8Yoh+gjzliwDbmR7hWmyNGknOBe4GbO89WAlBVh6vqyqq6HPhD4MnBw5YkSZLUJnMj/q8BB4HrO9UTrwBOVdUzwAPADyc5r1Ok44c7x4bS86OJVXUqyT7gd5N8rKq+ubRNkk3AXcxPcrtzybnvqqpnOyNiPwt8aMjYJUmSJKknSX6L+ZGt70xynPlKiBsBqupfAfcBfxV4ivkihO/tnDuZ5BeBz3Ru9cGqWq3oR0/6KiFfVUeSHAX2MP/o4VLXAO8AXp3kxs6xG6tqBviZJP8z86NwH6uqfz942JIkSZLaZLYammi3gm6JTVW9p8v5YoWn/6rqVuDWAUNbVtdErKq2LNnfvUrbA8CBFc79DPAz/QYoSZIkSZNmrBdVbqLi4caG1o22+qIkTaCmao5ZWmplravrtsbS0BtUfghXUg29xX6UhzfnX5aLDJSlJNnOKx9NPF1VO4cPSZIkSZIm20CJWFUdA6YajkWSJEnShGqosuHEaGhJOUmSJElSr7omYklmk8wkeSzJ3Um2rtJ2KsnhJI8neTTJtQvO7UryXzr3+o9J3tjUFyFJkiRpvM1WjXRrm54XdK6qS4GTrL6g84vA9VX1VuBq4KMLErePAX+zszj0vwFuHiJuSZIkSWqtfh9NPAxctNLJqnqiqp7svD4BPAtccOY0cG7n9XcAJ5a7R5K9SaaTTH/8wAt9hidJkiRpHM1RI93apudiHUk2ALuAT/TY/nJgE/CFzqH3AfcleQl4Abhiueuqaj+wH+D0M29o3zsqSZIkSV30MiK2OckM8BxwPnCo2wVJLmS+vP17q761hPZPAX+1qi4Gfh34Z4OFLEmSJKltZqmRbm3T8xwxYBvzI1yrzREjybnAvcDNVfVw59gFwPdV1R91mn0S+P6Bo5YkSZKkFut5jlhVnQL2ATcl2bhcmySbgLuA26vqzgWnvgp8R5Lv7ez/EPC5wUKWJEmS1DbOEVusrwWdq+pIkqPAHuYfPVzqGuAdwKuT3Ng5dmNVzST5O8DvJJljPjH724OHLUmSJEnt1TURq6otS/Z3r9L2AHBghXN3MT9atqa+WS83cp+N6StnXVFT8UiSGtC+X6D2ZlK/rknUwrWP2ia+xWOjjWt9jVK/5eslSZIkSUMaaJgnyXZe+Wji6araOXxIkiRJkibNXPcmZ5WBErGqOgZMNRyLJEmSpAnVxhLzozTUo4lJZpPMJHksyd1JtnZp/91J/l2SzyX5bJLXD9O/JEmSJLXRsHPEXqqqqaq6FDhJlzXGgNuBW6rqLcDlwLND9i9JkiSpBWZrtFvbNFms4zBw0Uonk/yPwLdV1SGAqvp6Vb3YYP+SJEmS1AqNJGJJNgC7gIOrNPte4Pkk/zbJkSS3dK5beq+9SaaTTH/8wAtNhCdJkiRpnc2NeGubYRfH2pxkBng98AhwqEtfVwJvA/4E+CRwI/CJhY2qaj+wH+D0M29o4SCjJEmSJK2ukTliwDZgE6vPETsOHKmqL1bVy8CngT8/ZP+SJEmSWmCWjHRrm0YeTayqU8A+4KYkG1do9hngvCQXdPbfCXy2if4lSZIkqU0aK9ZRVUeAo8CeFc7PAjcBDyY5BgT4tab6lyRJkjS+5mq0W9sMNUesqrYs2d/dpf0h4LJh+pQkSZKkthu2WMdZ45v1ciP32Zhm3vKm4pE0Ak39Wu6c9j3v3jpNvcXj9ptYPzqjlzF7k2vcPoRa0Vn8rWrjPK5RajwRS7IduGPJ4dNVtbPpviRJkiSpjRpPxKrqGDDV9H0lSZIktZcjYos1VqxDkiRJktSboUbEkswCxzr3eRq4rqqe73LNucDngLuq6gPD9C9JkiSpHebKEbGFGlnQuaouBU6y+oLOZ/wi8B+G7FeSJEmSWqvJOWKH6VKaPslfAP4ccD+wo8G+JUmSJI0x54gt1sgcsSQbgF3AwVXanAP8U+Bnutxrb5LpJNMfP/BCE+FJkiRJ0lgZdkRsc5IZ4PXAI8ChVdr+L8B9VfXlrLL2RlXtB/YDnH7mDWfxSguSJEnS5Ji1TuAijcwRA7YBm1h9jtj/BHwgyR8Dvwxcn+TDQ/YvSZIkSa3TyByxqjqVZB/wu0k+VlXfXKbN3zzzOsmNwI6q+odN9C9JkiRpvFk1cbHGxger6ghwFNjT1D0lSZIkTYZZMtKtbYYaEauqLUv2d/d43W8AvzFM3231zXq5kftsTDMFL5uKR5JaaVJnIk/q1zVOyjdZA2pfvqARabJ8vSRJkiQta7Ys1rFQ44lYku3AHUsOn66qnU33JUmSJElt1HgiVlXHgKmm7ytJkiSpveYsX7/IUO9GktkkM0keS3J3kq2rtN2W5JFO+8eT/L1h+pYkSZKkthp2ROzMOmIkuY35dcQ+tELbZ4Dvr6rTSbYAjyU5WFUnhoxBkiRJ0phrY2XDUWpyfPAwcNFKJ6vqG1V1urP7qob7liRJkqTWaCQZSrIB2AUc7NLudUkeBb4M/NJyo2FJ9iaZTjL98QMvNBGeJEmSpHU2W+eMdGubYR9N3JxkBng98AhwaLXGVfVl4LIkrwU+neRTVfXflrTZD+wHOP3MG1ykQ5IkSdLEGTZ1PDNHbBuwifk5Yl11RsIeB64csn9JkiRJLTBHRrq1TSNjeFV1CtgH3JRk43JtklycZHPn9XnA24HPN9G/JEmSJLVJY+uIVdWRJEeBPbxyQWeAtwD/NEkBAX65s+aYJEmSpAk3a62+RYZKxKpqy5L93au0PQRcNkx/kiRJkjQJGhsR09r6Zr3cyH02ZviPQFOxSBPjnPY9p37WaupbNW6lpfwItkca+mbVuH0IJ1Bj7/HZ+we0jZUNR6nxRCzJdl75aOLpqtrZdF+SJEmS1EaNJ2KdeV9TTd9XkiRJUnvNOUdsEd8NSZIkSVpjXROxJLNJZpI8luTuJFtXaTuV5HCSx5M8muTaBecuSfJHSZ5M8skkm5r6IiRJkiSNt9nKSLe26WVE7KWqmqqqS4GTrL5o84vA9VX1VuBq4KMLErdfAj5SVW8Cvgr85BBxS5IkSVJr9fto4mHgopVOVtUTVfVk5/UJ4FnggiQB3gl8qtP0NuDd/YcrSZIkqY1mOWekW9v0HHGSDcAu4GCP7S8HNgFfAF4NPF/1rTrnx1khoUuyN8l0kumPH3ih1/AkSZIkjbG5OmekW9v0UjVxc5IZ4PXAI8ChbhckuZD5EvY3VNVcZ0RsqWUXY6iq/cB+gNPPvMFFMSRJkiRNnJ7niAHbmB/hWm2OGEnOBe4Fbq6qhzuH/zuwNfnW6sEXAycGC1mSJElS2/ho4mI9R1xVp4B9wE1JNi7XplMJ8S7g9qq6c8G1Bfw+8OOdQzcAvzto0JIkSZLUZn2ljlV1BDgK7FmhyTXAO4AbOyXvZ5KcWdz5Z4GfTvIU83PGPjFgzJIkSZJaxvL1i3WdI1ZVW5bs716l7QHgwArnvghc3m+AGq1vfqt+ynA2ppfphqtrKhZp3c01NL31nPb9o9I6kzoTeVK/rklUfrNaY9mSB9Lghv/pWWe9JpIwSZIkTba5Fs7jGqWBfoJOsp35qogLna6qncOHJEmSJEmTbaBErKqOAVNdG0qSJEkSMNvCtb5Gqeu7kWS2U3TjsSR3J9m6StupJIeTPJ7k0STXLjj3m0k+37nPrStVXpQkSZKkSdfzOmJVdSlwktXXEXsRuL6q3gpcDXx0QeL2m8Cbge3AZuB9g4ctSZIkqU3myEi3tun30cTDwGUrnayqJxa8PpHkWeAC4Pmquu/MuST/mflFnSVJkiTprNPzg5pJNgC7gIM9tr8c2AR8YcnxjcB1wP0rXLc3yXSS6Y8feKHX8CRJkiSNsdk6Z6Rb2/QyIrY5yQzweuAR4FC3C5JcyHxVxRuqam7J6X8J/GFVPbTctVW1H9gPcPqZN7i4hiRJkqSJ0/McMWAb8yNcq80RI8m5wL3AzVX18JJzv8D8o4o/PVi4kiRJktpolnNGurVNzxFX1SlgH3DTShUPk2wC7gJur6o7l5x7H3AV8J5lRskkSZIk6azRV+pYVUeAo8CeFZpcA7wDuLFT8n4myZn1xv4V8OeAw53jPz9o0JIkSZLaZa4y0q1tus4Rq6otS/Z3r9L2AHBghXMDLR4tSZIkSZPG5EhD+2a93Mh9NqaZj2NT8UgDO6d9v5XThBmnj2BTZbfG6WuSNJA2zuMapYF+8k2ynfmqiAudrqqdw4ckSZIkSZNtoESsqo4BU10bSpIkSRIw18K1vkbJd0OSJEmS1thQk3KSzALHOvd5Griuqp5foe0U8DHgXGAW+FBVfXKY/iVJkiS1w6yTPRcZdkTspaqaqqpLgZOsvtjzi8D1VfVW4Grgo0m2Dtm/JEmSpBaYq3NGurVNk1UTDwOXrXSyqp5Y8PpEkmeBC4BlR9AkSZIkaVI1kjom2QDsAg722P5yYBPwhWXO7U0ynWT64wdeaCI8SZIkSetslox0a5thR8Q2J5kBXg88AhzqdkGSC5kvfX9DVc0tPV9V+4H9AKefeUNTq49IkiRJOssluRr458AG4ONV9eEl5z8C/KXO7rcD31VVWzvnztTHAPiTqnrXMLEMm4i9VFVTSb4DuIf5OWK/slLjJOcC9wI3V9XDQ/YtSZIkqSXWex5X5ym+fwH8EHAc+EySg1X12TNtquqnFrT/X4G3LbjFS1XV2BJejbwbVXUK2AfclGTjcm2SbALuAm6vqjub6FeSJEmSenQ58FRVfbGqvgH8NvCjq7R/D/BbowqmsbS0qo4AR4E9KzS5BngHcGOSmc7motCSJEnSWWC2zhnptrDWRGfbuySEi4AvL9g/3jn2Ckm2AZcA/37B4T/Tue/DSd497Psx1KOJVbVlyf7uVdoeAA4M058kSZIkLWdhrYkVLFfRY6WaFHuAT1XV7IJj392p/v4G4N8nOVZVryg+2Ksmy9dLQ/lmvdzIfTammY/1HK+oJSP1Zq6hOkPntK8CVFcT+CWpC7/nkjrm1v8vhOPA6xbsXwycWKHtHpaskVxVJzr//2KSP2B+/tjAiVjjM+aSbF/w6OGZ7Y+a7keSJEmS+vAZ4E1JLunUr9jDMstvJfkfgPOYXyf5zLHzkryq8/o7gbcDn116bT8aHxGrqmOAc78kSZIkfcvsOldNrKqXk3wAeID58vW3VtXjST4ITFfVmaTsPcBvV9XCR1zeAvzrJHPMD2Z9eGG1xUF0TcQW1Mv/NuBp4Lqqen6FtlPAx4BzgVngQ1X1yc653wB+ADjVaX5jVc0ME7wkSZIk9aqq7gPuW3Ls55fs/+/LXPefgO1NxtLLiNi36uUnuY35ZyU/tELbF4Hrq+rJJK8FHknywILE7Weq6lNDRy1JkiSpVeZq3eeIjZV+H008DFy20smqemLB6xNJngUuAJYdQZMkSZKks1HPD2p2VqLexTIT2lZofzmwicWVRD6U5NEkHzkz2W2Z675V///jB17oNTxJkiRJY2yWc0a6tU0vEW9OMgM8B5wPHOp2QZILgTuA91bVmRrgPwe8GfiLnfv87HLXVtX+qtpRVTve97fO7SE8SZIkSWqXXhKxM3PEtjE/wvX+1RonORe4F7i5qh4+c7yqnql5p4FfBy4fPGxJkiRJbTJXGenWNj2P4VXVKWAfcFOSjcu16dTjvwu4varuXHLuws7/A7wbeGzQoCVJkiSpzfoq1lFVR5IcZX7xszuWaXIN8A7g1Ulu7Bw7U6b+N5NcAASYAf7ewFFLkiRJapW5Fs7jGqWuiVhVbVmyv3uVtgeAAyuce2ff0UmSJEnSBOq3fP2a2hCzZvVvjrnujXpwjr+10aDOad9z6mum1juAs8Q4fQT9nkvqmG3hPK5RGigRS7KdVz6aeLqqdg4fkiRJkqRJ08aCGqM0UCJWVceAqYZjkSRJkqSzwlg/mihJkiRpMsyV0z4WavTdSPL1Vc5tS/JIkpkkjyexaqIkSZKks9Jajog9A3x/VZ1OsgV4LMnBqjqxhjFIkiRJWgezY1VJaP2tWSJWVd9YsPsqGh6NkyRJkqS2WNNkKMnrkjwKfBn4peVGw5LsTTKdZPrX7nhhLcOTJEmSNCJzlZFubbOmxTqq6svAZUleC3w6yaeq6r8tabMf2A/w8lfe6OojkiRJkibOulRNrKoTSR4HrgQ+tR4xSJIkSVo7Vk1cbM3ejSQXJ9nceX0e8Hbg82vVvyRJkiSNi7UcEXsL8E+TFBDglzsLQ0uSJEmacHNWTVyk0USsqrascu4QcFmT/UmSJElSG63LHLFe/ZXXvm29Q5CG9sCJo+sdwti6+l3b1zsErbHP79283iHoLPe6+7Y2cp+MWTmxcSoYl7lm3pxzXh6zN7khmV3vCNbP7Dh9UMdA44lYku3AHUsOn66qnU33JUmSJElt1Hgi1pn3NdX0fSVJkiS1l1UTF/PdkCRJkqQ11mgiluTrq5ybSnI4yeNJHk1ybZN9S5IkSRpfc5WRbm2zlsU6XgSur6onk7wWeCTJA1X1/BrGIEmSJEnrbs0Ssap6YsHrE0meBS4AFiViSfYCewHezJ/n4rxhrUKUJEmSNCKuI7bYuswRS3I5sAn4wtJzVbW/qnZU1Q6TMEmSJEmTaM3XEUtyIfPl7W+oqrm17l+SJEnS2mvjPK5RWtNELMm5wL3AzVX18Fr2LUmSJGn9WL5+sTV7N5JsAu4Cbq+qO9eqX0mSJEkaN2s5InYN8A7g1Ulu7By7sapm1jAGSZIkSevARxMXazQRq6otq5w7ABxosj9JkiRBar0jkNSvNS/WIUmSJOnsY/n6xRpPxJJsZ74q4kKnq2pn031JkiRJUhs1nohV1TFgqun7SpIkSWov54gt1rVqYpLZJDNJHktyd5KtXdrfn+T5JPcsOf6bST7fuc+tSTYOG7wkSZIktVEv5etfqqqpqroUOAm8v0v7W4Drljn+m8Cbge3AZuB9/QQqSZIkqb3mKiPd2qbfdcQOAxet1qCqHgS+tszx+6oD+M/AxX32LUmSJEkToedELMkGYBdwcJgOO48kXgfcv8L5vUmmk0wfry8O05UkSZKkMeGI2GK9JGKbk8wAzwHnA4eG7PNfAn9YVQ8td7Kq9lfVjqracXHeMGRXkiRJkjR+ep4jBmwDNtF9jtiKkvwCcAHw04PeQ5IkSVL7OCK2WM+PJlbVKWAfcNMgFQ+TvA+4CnhPVc31e70kSZIkTYq+inVU1RHgKLBnpTZJHgLuBHYlOZ7kqs6pfwX8OeBwpxz+zw8YsyRJkqSWmSMj3dqm64LOVbVlyf7uLu2vXOF444tHS5IkSVIbmRxJ0hn9LujRFk08DD6p7400oKamo6Sauc84TY9p6mvS5GnjPK5RGigRS7IduGPJ4dNVtXP4kCRJkiRpsg2UiFXVMWCq4VgkSZIkTShHxBbzYRNJkiRJWmNDzRFLMgsc69znaeC6qnp+hbZ/CfjIgkNvBvZU1aeHiUGSJEnS+HNEbLFhi3WcWeyZJLcxv9jzh5ZrWFW/T+dxxiTnA08B/27I/iVJkiS1gInYYk0+mngYuKjHtj8O/F5Vvbj0RJK9SaaTTB+vLzYYniRJkiSNh0YSsSQbgF3AwR4v2QP81nInqmp/Ve2oqh0X5w1NhCdJkiRpnVVlpFvbDJuIbU4yAzwHnA8c6nZBkguB7cADQ/YtSZIkSa00bCJ2Zo7YNmAT83PEurkGuKuqvjlk35IkSZJaYo6MdGubRh5NrKpTwD7gpiQbuzR/Dys8lihJkiRJZ4PGinVU1RHgKPPzv5aV5PXA64D/0FS/kiRJksbfXGWkW9sMVb6+qrYs2d/dpf0f03tlRUlaW3PrHcAY872RJKlRw64jJkmSJEldtbGy4Sg1nogl2Q7cseTw6ara2XRfkiRJktRGjSdiVXUMmGr6vpIkSZLaq43zuEapa7GOJLNJZpI8luTuJFtXaTuV5HCSx5M8muTaZdr8apKvDxu4JEmSJLVVL1UTX6qqqaq6FDjJ6muFvQhcX1VvBa4GProwcUuyA1gxkZMkSZI0maoy0q1t+i1ff5hVqh5W1RNV9WTn9QngWeACgCQbgFuAfzBYqJIkSZI0GXpOxDqJ1C7gYI/tLwc2AV/oHPoAcLCqnuly3d4k00mmj9cXew1PkiRJ0hhzHbHFeknENieZAZ4DzgcOdbsgyYXMV058b1XNJXkt8BPAr3a7tqr2V9WOqtpxcd7QQ3iSJEmS1C49zxEDtjE/wrXaHDGSnAvcC9xcVQ93Dr8NeCPwVJI/Br49yVMDRy1JkiSpVapGu7VNz48mVtUpYB9wU5KNy7VJsgm4C7i9qu5ccO29VfWaqnp9Vb0eeLGq3jhc6JIkSZLUTn0V66iqI8BRYM8KTa4B3gHc2Cl5P5PENcUkSZKks9wcGenWNl0XdK6qLUv2d6/S9gBwoN97SpIkSdLZpGsitp6+8SOXr3cI0tCuftf29Q5hbN1/sOvvbTRhrvrxG9Y7hMWamlSQMftNbBNf17h9TQ05+ZZvX+8QFklDn8Eao+/Xt/+/Lzdyn4zZnJ/GivKN0fdqrbVxra9RGigRS7Kd+aqIC52uqp3DhyRJkiRJk22gRKyqjgHO/ZIkSZLUkzau9TVKY/1ooiRJkqTJ0MYS86PUtWpiktlO9cPHktydZGuX9vcneT7JPUuOJ8mHkjyR5HNJ9g0bvCRJkiS1Uc8LOlfVpcBJuizoDNwCXLfM8RuB1wFvrqq3AL/dT6CSJEmS2qsqI93apq91xIDDwEWrNaiqB4GvLXPq7wMfrKq5Trtn++xbkiRJkiZCz4lYkg3ALuDggH19D3Btkukkv5fkTSv0s7fTZvqZLz08YFeSJEmSxokjYov1kohtTjIDPAecDxwasK9XAX9aVTuAXwNuXa5RVe2vqh1VtePCbVcM2JUkSZIkja+e54gB24BNdJ8jtpLjwO90Xt8FXDbgfSRJkiS1zFxlpFvb9PxoYlWdAvYBNyXZOEBfnwbe2Xn9A8ATA9xDkiRJklqvr2IdVXUEOArsWalNkoeAO4FdSY4nuapz6sPAjyU5Bvxj4H2DhSxJkiSpbapGu7VN1wWdq2rLkv3dXdpfucLx54Ef6Ss6SZKkCVZp3+NUbdPCJ9Y0QkmuBv45sAH4eFV9eMn5G5lfjuv/6Rz6P6rq451zNwA3d47/o6q6bZhYuiZikiRJkjSs9a5s2KkC/y+AH2K+fsVnkhysqs8uafrJqvrAkmvPB34B2AEU8Ejn2q8OGk+/64idCWR7kpkl2x8NGoQkSZIkjdjlwFNV9cWq+gbw28CP9njtVcChqjrZSb4OAVcPE8xAI2JVdQyYGqZjSZIkSWeP9R4RAy4Cvrxg/ziwc5l2P5bkHcwXF/ypqvryCtdeNEwwA42IrSTJ17ucvz/J80nuabJfSZIkSWe3JHuTTC/Y9i5tssxlS8t83A28vqouA/4v4Mw8sF6u7ctazxG7Bfh24O+ucb+SJEmS1tGoCxtW1X5g/ypNjgOvW7B/MXBiyT2eW7D7a8AvLbj2B5dc+wcDhgo0PCLWTVU9CHxtLfuUJEmSJOAzwJuSXJJkE/NLch1c2CDJhQt23wV8rvP6AeCHk5yX5DzghzvHBjZ2VRM7Q4h7Ad60/a9z4bYr1jkiSZIkScNa7zliVfVykg8wn0BtAG6tqseTfBCYrqqDwL4k7wJeBk4CN3auPZnkF5lP5gA+WFUnh4ln7BKxhUOKP7D7lhYuzSZJkiRpHFXVfcB9S479/ILXPwf83ArX3grc2lQsY5eISZIkSZpADrEssqZzxCRJkiRJazwiluQh4M3AliTHgZ+sqqEmuUmSJEkaf+s9R2zcNJqIVdWWLuevbLI/SZIkSe1QPpq4iHPEpLNNEw8kzzVwD0mjE3/rLJ2Rhn74dzBHTWs8EUuyHbhjyeHTVbWz6b4kSZIktYOPJi7WeCJWVceAqabvK0mSJEmTwkcTJUmSJI2eI2KLdJ0tkmQ2yUySx5LcnWRrl/b3J3k+yT1Lju9K8l869/qPSd44bPCSJEmS1Ea9TNt/qaqmqupS4CTw/tI9VPcAACAASURBVC7tbwGuW+b4x4C/WVVTwL8Bbu4rUkmSJEmtVTXarW36rZ92GLhotQZV9SDwteVOAed2Xn8HcKLPviVJkiRpIvQ8RyzJBmAX8IkB+3ofcF+Sl4AXgCtW6GcvsBfgTdv/OhduW7aZJEmSpDZp4ajVKPUyIrY5yQzwHHA+cGjAvn4K+KtVdTHw68A/W65RVe2vqh1VtcMkTJIkSdIk6nmOGLAN2ET3OWKvkOQC4Puq6o86hz4JfH+/95EkSZLUTlUZ6dY2Pc8Rq6pTwD7gpiQb++znq8B3JPnezv4PAZ/r8x6SJEmSNBH6Wkesqo4kOQrsAe5Yrk2Sh4A3A1uSHAd+sqoeSPJ3gN9JMsd8Yva3hwtdkiRJUms4R2yRrolYVW1Zsr+7S/srVzh+F3BXX9FNorkx+wSe09AwbhNf1zjFAs3FM4nOAebWOwhJar+0seZ2N2no389JfG+kBfoaEZOWNW7JpUbPJEySJPWpjfO4RmmgRCzJdl75aOLpqto5fEiSJEmSNNkGSsSq6hgw1XAskiRJkiaVD1Et0rVqYpLZJDNJHktyd5KtXdrfn+T5JPcsOf4bSZ7u3GsmiYmcJEmSpLNSz+uIVdWlwEm6ryN2C3DdCud+pnOvqaqa6SdQSZIkSW2WEW/t0vM6Yh2HgYtWa1BVDwJfGzgiSZIkSZpwPSdiSTYAu4CDQ/T3oSSPJvlIklet0M/eJNNJpp/50sNDdCVJkiRpbNSIt5bpJRHbnGQGeA44Hzg0YF8/x/xCz3+xc5+fXa5RVe2vqh1VtePCbVcM2JUkSZKksWIitkjPc8SAbcAmus8RW1ZVPVPzTgO/Dlw+yH0kSZIkqe16fjSxqk4B+4Cbkmzst6MkF3b+H+DdwGP93kOSJElSS1VGu7VMX8U6quoIcBTYs1KbJA8BdwK7khxPclXn1G8mOQYcA74T+EeDhSxJkiRJ7dZ1Qeeq2rJkf3eX9leucPyd/YUmSZIkaVJUC+dxjVLXREwNO6d9w6ZdjdvXNG7xSJJ0Nhmzn7bH7Ym1cYtH62egRCzJduCOJYdPV9XO4UOSJEmSNHHGK0dfdwMlYlV1DJhqOBZJkiRJOiv4aKIkSZKk0fO5zEW6Vk1MMptkJsljSe5OsnWVtlNJDid5PMmjSa5dcO6hzn1mkpxI8ummvghJkiRJapNeRsTOLOhMktuYX9D5Qyu0fRG4vqqeTPJa4JEkD1TV8wurKSb5HeB3h4xdkiRJUkvEOWKL9LWOGHAYuGilk1X1RFU92Xl9AngWuGBhmyR/Fngn4IiYJEmSpLNSz4lYkg3ALuBgj+0vBzYBX1hy6q8BD1bVCytctzfJdJLpZ770cK/hSZIkSRpnNeKtZXpJxDYnmQGeA84HDnW7IMmFzJe3f29VzS05/R7gt1a6tqr2V9WOqtpx4bYreghPkiRJktqll0TszByxbcyPcL1/tcZJzgXuBW6uqoeXnHs1cHnnvCRJkqSzRWW0W8v0/GhiVZ0C9gE3Jdm4XJskm4C7gNur6s5lmvwEcE9V/ekgwUqSJEnSJOirWEdVHQGOAntWaHIN8A7gxgWl6hcu/LyHVR5LlCRJkjShnCO2SNfy9VW1Zcn+7lXaHgAOrHL+B/sJTpIkSZImUS/riEmaJEvL50iSWq8yRvNjmoqlmhniaGrtqqamIJ3Va2mdzV/7MgZKxJJsZ74q4kKnq2rn8CFJkiRJ0mQbKBGrqmPAVNeGkiRJkgSOiC3RtVhHktlO0Y3HktydZGuX9vcneT7JPUuOfyLJ0SSPJvlUki0r3UOSJEnShLF8/SI9ryNWVZcCJ+myjhhwC3DdMsd/qqq+r6ouA/4E+EB/oUqSJEnSZOirfD1wGLhotQZV9SDwtWWOvwCQJMBmHJyUJEmSzhqp0W5t03MilmQDsAs4OGhnSX4d+ArwZuBXV2izN8l0kulnvvTwoF1JkiRJ0tjqJRHbnGQGeA44Hzg0aGdV9V7gtcDngGtXaLO/qnZU1Y4Lt10xaFeSJEmSxokLOi/S8xwxYBuwie5zxFZVVbPAJ4EfG+Y+kiRJktRWPT+aWFWngH3ATUk29tNJ5r3xzGtgN/Bf+7mHJEmSJE2Kvop1VNUR4CiwZ6U2SR4C7gR2JTme5CogwG1JjgHHgAuBDw4ctSRJkiS1WNcFnatqy5L93V3aX7nCqbf3EZckSZKkCdLGyoaj1DURkzRh+l20YjlzDdxD0uhUAz/tpH2Lo7ZR+T6fdVq47rBGZKBELMl24I4lh09X1c7hQ5IkSZI0ccxCFxkoEauqY8BUw7FIkiRJ0lnBRxMlSZIkjZ5zxBYZKhFLMst8FcRvA54Grquq53toD/AnVfWuYfqXJEmSpDYadtr+S1U1VVWXAifpvtjzmfZTJmGSJEnSWaRGvLVME/XTzjgMXNTg/SRJkiRpIjWSiCXZAOwCDnZp+meSTCd5OMm7V7jX3k6b6We+9HAT4UmSJElaZ6nRbm0zbCK2OckM8BxwPnCoS/vvrqodwN8APprke5Y2qKr9VbWjqnZcuO2KIcOTJEmSpPHTyBwxYBuwiS5zxKrqROf/XwT+AHjbkP1LkiRJagPniC3SyKOJVXUK2AfclGTjcm2SnJfkVZ3X3wm8HfhsE/1LkiRJUps0Vqyjqo4AR4E9KzR5CzCd5Cjw+8CHq8pETJIkSTobOCK2yFDriFXVliX7u1dp+5+A7cP0J0mSJEmTYKhEbNRmN2a9Q1AbNfWxaeg3K5/fu7mZG02gq378hvUOQWvsgU/dtt4h6Cx31V+/vpH7pFr46/cuKs38A3r6O1/VyH2qyUWWGtBYVb7J++j0rI2VDUep8UQsyXbgjiWHT1fVzqb7kiRJktQS5SDLQo0nYlV1DJhq+r6SJEmSNCnG+tFESZIkSRPCRxMX6fr0bZLZJDNJHktyd5KtXdrfn+T5JPcsOf5Q5z4zSU4k+fSwwUuSJElSG/UyDfKlqpqqqkuBk3RZtBm4Bbhu6cGqurJznyngMPBv+45WkiRJUiulRru1Tb/1aA4DF63WoKoeBL620vkkfxZ4J7DsiFiSvUmmk0x/5emH+wxPkiRJksZfz4lYkg3ALuDgkH3+NeDBqnphuZNVtb+qdlTVjtdccsWQXUmSJEkaCy7ovEgvidjmJDPAc8D5wKEh+3wP8FtD3kOSJEmSWqvnOWLANmAT3eeIrSjJq4HLgXsHvYckSZKk9nGO2GI9P5pYVaeAfcBNSTYO2N9PAPdU1Z8OeL0kSZIktV5fxTqq6ghwFNizUpskDwF3AruSHE9y1YLTe/CxREmSJOns4xyxRbou6FxVW5bs7+7S/spVzv1gz5FJkiQ1LNXCn9bWyLi9N009alZp5j5S07omYpIkSZI0tPHK9dfdQIlYku3AHUsOn66qncOHJEmSJEmTbaBErKqOAVMNxyJJkiRpQrWxsuEodS3WkWQ2yUySx5LcnWTrKm2nkhxO8niSR5Ncu+BcknwoyRNJPpdkX1NfhCRJkiR1k+TqJJ9P8lSSf7jM+Z9O8tlOLvNgkm0Lzp3Ji2aSHBw2ll5GxM6sI0aS25hfR+xDK7R9Ebi+qp5M8lrgkSQPVNXzwI3A64A3V9Vcku8aNnhJkiRJ6kWSDcC/AH4IOA58JsnBqvrsgmZHgB1V9WKSvw/8E+DM4NK38qIm9FW+HjgMXLTSyap6oqqe7Lw+ATwLXNA5/feBD1bVXOf8s/2HK0mSJEkDuRx4qqq+WFXfAH4b+NGFDarq96vqxc7uw8DFowqm50Ssk0HuAnoahktyObAJ+ELn0PcA1yaZTvJ7Sd60wnV7O22mv/L0w72GJ0mSJGmcrf86YhcBX16wf5xVBpmAnwR+b8H+n+nkKQ8neXdPPa6il0Rsc5IZ4DngfOBQtwuSXMh8VcX3nhkBA14F/GlV7QB+Dbh1uWuran9V7aiqHa+55IpevgZJkiRJZ7mFAzqdbe/SJstctmwKl+RvATuAWxYc/u5OLvM3gI8m+Z5h4u0lETvzLOQ25ke43r9a4yTnAvcCN1fVwiGt48DvdF7fBVzWf7iSJEmS2ig12m3hgE5n278khOPM16w442LgxCviTP4y8L8B76qq02eOd6ZeUVVfBP4AeNsw70fPjyZW1SlgH3BTko3LtUmyifkk6/aqunPJ6U8D7+y8/gHgif7DlSRJktRK6/9o4meANyW5pJO37GHJtKskbwP+NfNJ2LMLjp+X5FWd198JvB1YWOSjb30V66iqI8DRTtDLuQZ4B3DjgtKOZyqLfBj4sSTHgH8MvG/AmCVJkiSpL1X1MvAB4AHgc8D/WVWPJ/lgknd1mt0CbAHuXFKm/i3AdJKjwO8DH15SbbFvXcvXV9WWJfu7V2l7ADiwwrnngR/pN0BJkiRJE2AMFnSuqvuA+5Yc+/kFr//yCtf9J2B7k7H0so5Yuy03JW8QTX1wJjGeMfhDpZYqPzySpOVVUz8zjRv/7VPHQIlYku3MV0Vc6HRV7Rw+JEmSJEmTJuagiwyUiFXVMaCxVaUlSZIk6Wwy+Y8mSpIkSVp/jogt0rVqYpLZTsWQx5LcnWTrKm2nkhxO8niSR5Ncu+DcB5I8laQ6JR8lSZIk6azU84LOVXUpcJLVF3R+Ebi+qt4KXM38itNnErf/G/jLwJeGCViSJElS+4x6Qee26WsdMeAwcNFKJ6vqiap6svP6BPAscEFn/0hV/XG3DpLsTTKdZPorTz/cZ3iSJEmSNP56TsSSbAB2sWT16VXaXw5sAr7QT0BVtb+qdlTVjtdcckU/l0qSJEkaVzXirWV6ScQ2J5kBngPOBw51uyDJhcyXt39vVc0NF6IkSZIkTZae54gB25gf4VptjhhJzgXuBW6uKp8tlCRJkuSI2BI9P5pYVaeAfcBNSTYu1ybJJuAu4PaqurOZECVJkiRpsvRVrKOqjgBHgT0rNLkGeAdwY6fk/UySKYAk+5IcBy4GHk3y8SHiliRJktQiVk1crOuCzlW1Zcn+7lXaHgAOrHDuV4Bf6TfAoY3bN8V4pP9fst4RSDrL1Jj9vZMan3+Ix+29mVi+z+romohJkiRJ0tDG5/cOY2GgRCzJduarIi50uqp2Dh+SJEmSJE22gRKxqjoGTDUciyRJkqRJ5YjYIl2LdSSZ7RTdeCzJ3Um2rtJ2KsnhJI8neTTJtQvOfSLJ0c7xTyXZstJ9JEmSJGmS9byOWFVdCpxk9XXEXgSur6q3AlcDH12QuP1UVX1fVV0G/AnwgWEClyRJktQeVk1crN9HEw8Dl610sqqeWPD6RJJngQuA56vqBYAkATbj4KQkSZJ09vCn/0V6XkcsyQZgF3Cwx/aXA5uALyw49uvAV4A3A7+6wnV7k0wnmf7K0w/3Gp4kSZIktUYvidjmJDPAc8D5wKFuFyS5kPmqiu+tqrkzx6vqvcBrgc8B1y53bVXtr6odVbXjNZdc0UN4kiRJksadjyYu1vMcMWAb8yNcq80RI8m5wL3AzVX1iiGtqpoFPgn8WP/hSpIkSVL79fxoYlWdAvYBNyXZuFybJJuAu4Dbq+rOBceT5I1nXgO7gf86TOCSJEmSWqRGvLVMz4kYQFUdAY4Ce1Zocg3wDuDGTsn7mSRTQIDbkhwDjgEXAh8cPGxJkiRJaq+uVROrasuS/d2rtD0AHFjh9Nv7C02SJEnSxGjhqNUo9Vu+XpIkqbVS/iS4Et8baW0NlIgl2c58VcSFTlfVzuFDkiRJkjRpst4BjJmBErGqOgZMNRyLJEmSJJ0VfDRRkiRJ0uj59OsifVVN7CbJ17ucvyHJk53thib7liRJkqS2WLMRsSTnA78A7GA+H34kycGq+upaxSBJkiRpfcQRsUUaHRHr4irgUFWd7CRfh4Cr17B/SZIkSRoLa5mIXQR8ecH+8c6xRZLsTTKdZPorTz+8ZsFJkiRJGqEa8dYya5mILVex8hVvWVXtr6odVbXjNZdcsQZhSZIkSdLaWstE7DjwugX7FwMn1rB/SZIkSevFEbFF1jIRewD44STnJTkP+OHOMUmSJEk6q6xZ1cSqOpnkF4HPdA59sKpOrlX/kiRJktaPVRMXazQRq6otXc7fCtza8w2Xm1W2Xpr64DT1NY1TPOMUS5PGLZ5JVGP2N3L8pq9o3L5XkiS13JqNiEmSJEk6i/k7vUUaT8SSbAfuWHL4dFXtbLovSZIkSe3go4mLNZ6IVdUxYKrp+0qSJEnSpOhaNTHJbJKZJI8luTvJ1i7t70/yfJJ7lhy/JMkfJXkyySeTbBo2eEmSJEktYfn6RXopX/9SVU1V1aXASeD9XdrfAly3zPFfAj5SVW8Cvgr8ZF+RSpIkSdKE6HcdscPARas1qKoHga8tPJYkwDuBT3UO3Qa8u8++JUmSJLVUarRb2/SciCXZAOwCDg7Qz6uB56vq5c7+cVZI6JLsTTKdZPorTz88QFeSJEmSNN56ScQ2J5kBngPOBw4N0M9yi/Msm7dW1f6q2lFVO15zyRUDdCVJkiRp7DhHbJGe54gB24BNdJ8jtpz/DmxNcqZK48XAiQHuI0mSJEmt1/OjiVV1CtgH3JRkYz+dVFUBvw/8eOfQDcDv9nMPSZIkSS3miNgifRXrqKojwFFgz0ptkjwE3AnsSnI8yVWdUz8L/HSSp5ifM/aJwUKWJEmSpHbruqBzVW1Zsr+7S/srVzj+ReDyvqKTJEmSNBHaWNlwlLomYpI0Mlmujo/Gkt8rSZIaNVAilmQ7cMeSw6eraufwIUmSJEmaOI6ILTJQIlZVx4CphmORJEmSpLOCjyZKkiRJGrmUQ2IL9VU1cakks0lmkjyW5O4kW7u0vz/J80nuGaZfSZIkSWqzoRIxOos9V9WlwEm6L/Z8C3DdkH1KkiRJahvXEVtk2ERsocPARas1qKoHga812KckSZIktU4jiViSDcAu4GAD99qbZDrJ9Feefnj44CRJkiStu9Rot7YZNhHbnGQGeA44Hzg0bEBVtb+qdlTVjtdccsWwt5MkSZKksdPIHDFgG7CJ7nPEJEmSJJ2NnCO2SCOPJlbVKWAfcFOSjU3cU5IkSZImVWPFOqrqCHAU2LNSmyQPAXcCu5IcT3JVU/1LkiRJGl/OEVtsqAWdq2rLkv3dXdpf2V8HAwS1VBq4h1YXmvleNfUHqKnveQv/QEuSJI0tf7ZapMny9Tpb+YdKkiRJ6stQI2LLSbIduGPJ4dNVtbPpviRJkiS1QxsfHxylxhOxqjoGTDV9X0mSJEmaFF0fTUwym2QmyWNJ7k6ytUv7+5M8n+SeFc7/apKvDxqwJEmSpBayfP0ivcwRe6mqpqrqUuAk3dcKuwW4brkTSXYAqyZykiRJkjTp+i3WcRi4aLUGVfUg8LWlx5NsYD5J+wd99ilJkiSp5Sxfv1jPiVgnkdoFHBywrw8AB6vqmS797E0ynWT6K08/PGBXkiRJkjS+eknENieZAZ4DzgcO9dtJktcCPwH8are2VbW/qnZU1Y7XXHJFv11JkiRJGkdVo91apuc5YsA2YBPd54gt523AG4Gnkvwx8O1JnhrgPpIkSZLUej2Xr6+qU0n2Ab+b5GNV9c0+rr0XeM2Z/SRfr6o39heqJOn/a+/e4+Qo63yPf74JAYnhFlDxQARFd0WBDTgCrstFosKuB1EOIOeAgrIEVjGKhuMFXytHFlf0RHmpeHC8cXG9AN6CIogolz0mykgCCXLkIihRQCXIRTSYzO/8UTXS00zPVHU93VPV+b5fr3pNd/XTv+dXT1XX9NNV9ZSZmVlTNfE6rl4qNVhHRKwAbgKO7lRG0vXAJcACSWskHVwtRTMzMzMzs8Ey5RGxiJjT9vzQKcrvVzammZmZmZkNOB8RG6fwqYlmPafpTsDMzMzqItVpbOHvF1ZTXXXEJO0OXNQ2e11E7FM9JTMzMzMzGzQane4M6qWrjlhErALmJ87FzMzMzMxso+BTE83MzMzMrPd8jdg4pUZNbCdpg6SVklZLukzS1lOUPzsvu1rS66rUbWZmZmZm1lSVOmLkN3uOiN2AtUxys2dJrwL2IjulcR/gNElbVqzfzMzMzMwaQNHbqWmqdsRaLQN2mOT1FwDXRsT6iPgj2f3IDklYv5mZmZmZWSMk6YhJmgksAJZOUuwm4B8lzZa0HfAyYN4EsRZKGpE0ct9dy1OkZ2ZmZmZm0y2it1PDVO2IbS5pJfAAMBe4qlPBiPgecDnwI+DLZEfQ1k9QbjgihiJiaPtn71sxPTMzMzMzqwOfmjhekmvEgJ2ATZnkGjGAiDgrv6bsFWS37729Yv1mZmZmZmaFSDpE0s8l3SHp3RO8vpmkr+av/1jSzi2vvSef/3NJB1fNJcmpiRHxELAIWCxp1kRlJM2UtG3+eA9gD+B7Keo3MzMzM7Oaix5PU8gvpzoX+Eey8Sv+u6QXtBU7AXgwIp4LfAw4O3/vC4CjgReSjXPxqTxe15IN1hERK8iuAzu6Q5FZwPWSfgYMA8dGxJNOTTQzMzMzM+uBvYE7IuIXEfE48BXgsLYyhwEX5I8vBRZIUj7/KxGxLiLuAu7I43Wt0g2dI2JO2/NDJyn7Z7Kep5mZmZmZbWR6fR2XpIXAwpZZwxEx3PJ8B+CeludryG6rxURlImK9pIeAbfP5y9veO9mI8VOq1BHrtTl3/GG6UzCrbN7lk97nfKO2dtfZ052C9dnBh78hSRzVbHSskJLEqdtyDaIrvnHRdKcw8A44+cQkcWJGms8VqcKsT/P5nPH4aJI49mR5p2t4kiITbQ3tK7ZTmSLvLSV5R0zS7kD7Xm5dRLT3Ns3MzMzMbGMx/T82rWH87bN2BH7TocwaSZsAWwFrC763lOQdsYhYBcxPHdfMzMzMzKyCG4DnSXo28GuysS3+R1uZpcBxZLfaOgL4QUSEpKXAlyR9FPgvwPOAn1RJZsrBOiRtkLRS0mpJl0ma9DwrSVdI+oOkb7fNPyUf7jHyGzqbmZmZmdlGYrrvI5YPFHgKcCVwK3BxRNwi6QOSXp0X+xywraQ7gHcA787fewtwMfAz4ArgLRGxoUp7FDkiNnavMCRdQHavsLMmKf8RYDZwUtv8/wt8G7imfJpmZmZmZmbVRMTlwOVt8/615fGfgSM7vPcsJu8HlVJ2+PplTDE6SERcDTwywfwVEXF3yfrMzMzMzGwQTPN9xOqmcEcsv2HZArLzJntG0kJJI5JG7ll7Yy+rMjMzMzMzmxZFOmKbS1oJPADMBa7qZUIRMRwRQxExNG/uXr2syszMzMzM+mS6rxGrmyIdsbFrxHYCNiW7RszMzMzMzMy6VPjUxIh4CFgELJY0q3cpmZmZmZnZwBmN3k4NU2qwjohYAdxENub+hCRdD1wCLJC0RtLB+fxFktaQ3fzsZkmf7T5tMzMzMzOz5ppy+PqImNP2/NApyu/XYf7HgY+Xys7MzMzMzAZD8w5a9VSR+4hZSlKaOOEtuSnqdvFoJNoEzboh77vMrN+827Ga6qojJml34KK22esiYp/qKZmZmZmZ2aCp24/T062rjlhErALmJ87FzMzMzMxso+BTE83MzMzMrPd8evo4pUZNbCdpg6SVklZLukzS1lOU/7CkWyTdKunjUqoLpszMzMzMrM58Q+fxKnXEyG/2HBG7AWuZ5GbPkv4eeCmwB7Ab8GLggIr1m5mZmZmZNU7KUxOXkXWyOgngKcCmgIBZwP0J6zczMzMzs7pq4FGrXqp6RAwASTOBBcDSTmUiYhnwQ+DefLoyIm6dINZCSSOSRu5Ze2OK9MzMzMzMzGqlakdsc0krgQeAucBVnQpKei6wK7AjsANwkKT928tFxHBEDEXE0Ly5e1VMz8zMzMzM6kARPZ2aJsk1YsBOZKccdrxGDHgtsDwiHo2IR4HvAvtWrN/MzMzMzKxxkpyaGBEPAYuAxZJmdSj2K+AASZvkZQ4AnnRqopmZmZmZDaDRHk8Nk6QjBhARK4CbgKM7FLkUuBNYlZe7KSIuS1W/mZmZmZlZU1QaNTEi5rQ9P3SSshuAk6rUZ2ZmZmZmzdTE67h6KeXw9fWU6p7Rg7rhpGifQW2bAVWnGx6m2iFHze4NP4j/aOrWxnUziOvcbGB492U1lbwjJml34KK22esiYp/UdZmZmZmZWUP4N6txknfEImIVMD91XDMzMzMzs0Ex+KcmmpmZmZnZ9PNp3ONMOWqipA2SVkpaLekySVtPUf4KSX+Q9O22+QdJujGPc4EkdwLNzMzMzGyjVGT4+j9FxPyI2A1Yy+Q3bQb4CPD61hmSZgAXAEfncX4JHNdFvmZmZmZm1kCK3k5NU/Y+YsuAHSYrEBFXA4+0zd6WbMCO2/LnVwH/baL3S1ooaUTSyD1rbyyZnpmZmZmZWf0V7ohJmgksAJZ2Uc/vgVmShvLnRwDzJioYEcMRMRQRQ/Pm7tVFVWZmZmZmVjsRvZ0apkhHbHNJK4EHgLlkR7NKiYgAjgY+JuknZEfM1peNY2ZmZmZmNggKXyMG7ARsytTXiE0oIpZFxH4RsTdwHXB7N3HMzMzMzKx5NNrbqWkKn5oYEQ8Bi4DFkmaVrUjS0/O/mwHvAs4rG8PMzMzMzGwQlBqsIyJWADeRnWY4IUnXA5cACyStkXRw/tJpkm4FbgYui4gfdJmzmZmZmZk1ja8RG2fKe3lFxJy254dOUX6/DvNPA04rlV0KdVspzmejE0oTJ9WwrKnySSFUo2QGmNvZzKy6ZP/P04RpJn/tHKfs8PVmZmZmZmZW0ZRHxCYiaXfgorbZ6yJin+opmZmZmZnZoJHPxBqnq45YRKwC5ifOxczMzMzMbKNQ6dRESRskrZS0WtJlkraeovxxkm7Pp+Oq1G1mZmZmZg3iwTrGqXqN2J8iYn5E7AasZZJ7jEmaC7wf2AfYG3i/pG0q1m9mZmZmZtY4KQfrWAbsMMnrBwNXRcTaiHgQuAo4JGH9ZmZmZmZWV6M9nhomSUdM0kxgAbB0kmI7APe0PF/DBB033JEU7AAAGjRJREFUSQsljUgauWftjSnSMzMzMzMzq5WqHbHNJa0EHgDmkh3l6mSi2yY86WTOiBiOiKGIGJo3d6+K6ZmZmZmZWR0ooqdT0yS5RgzYCdiUSa4RIzsCNq/l+Y7AbyrWb2ZmZmZm1jhJTk2MiIeARcBiSbM6FLsSeKWkbfJBOl6ZzzMzMzMzs0HnURPHSTZYR0SsAG4Cju7w+lrgTOCGfPpAPs/MzMzMzGyj0tUNncdExJy254dOUf7zwOer1GlmZmZmZg3UwKNWvVSpI2ZmZlZGaKJxm8qr20XZg7pcZvYE+eNpiSXviEnaHbiobfa6iNgndV1mZmZmZtYQDbzXVy8l74hFxCpgfuq4ZmZmZmZmg8KnJpqZmZmZWc/59Ovxko2aCCDp0QJltpT0a0mfTFm3mZmZmZlZU0zHEbEzgWunoV4zMzMzM5suPiI2TtIjYlOR9CLgGcD3JimzUNKIpJF71t7Yv+TMzMzMzMz6pG8dMUkzgCXAaZOVi4jhiBiKiKF5c/fqT3JmZmZmZtZbEb2dGqafR8TeDFweEff0sU4zMzMzM7Pa6ec1Yi8B9pP0ZmAOsKmkRyPi3X3MwczMzMzMpkMDj1r1Ut86YhFxzNhjSccDQ+6EmZmZmZltJHxD53H6OliHmZmZmZmZJT4iFhFzCpY7Hzh/qnJ/2mmrihmZTT+N1uswvGqUzuzfrZ/uFHpDShMn1SkcqfJJYN12m013CraRO+DkE6c7hYF37XmfSRLnwJPSrKvRmWn2gTEj0b50Om4eVRO+ofN4PiJmZmZmZmbWZ8n75JJ2By5qm70uIvZJXZeZmZmZmTWEj4iNk7wjFhGrgPmp45qZmZmZmQ2Krk9NlBSSlrQ8XyzpjALvOyJ/71C3dZuZmZmZWcOMRm+nhqlyjdg64HBJ2xV9g6QtgEXAjyvUa2ZmZmZm1mhVOmLrgWHg1BLvORP4MPDnCvWamZmZmVnTRPR2apiqoyaeCxwjacpx5iXtCcyLiG9PUW6hpBFJI/f+cnnF9MzMzMzMzOqnUkcsIh4GLiQ73bAjSTOAjwHvLBBzOCKGImLomTvtWyU9MzMzMzOrCx8RGyfFfcTOAU4AnjpJmS2A3YBrJN0N7Ass9YAdZmZmZma2MarcEYuItcDFZJ2xTmUeiojtImLniNgZWA68OiJGqtZvZmZmZmYN4CNi46Q4IgawBCg8eqKZmZmZmdnGrOsbOkfEnJbH9wOzS7z3wG7rNTMzMzOzBmrgvb56qeuOWD+s30zTnYJZZTPWe6fTiQa1aep2ekSN8olE52Gk2nbC/2YaI9k6n+GV3msHnnRikjjXfPozSeLs/5aFSeKk2l+MzvQ2aJnkHTFJpwNHts2+JCLOSl2XmZmZmZk1RIxOdwa1krwjlne43OkyMzMzMzProNanJpqZmZmZ2YCo0anyddD12fqSQtKSlueLJZ0xSfmTJa2StFLSf0p6Qbd1m5mZmZmZNVmVy6bXAYdLKjps/ZciYveImA98GPhohbrNzMzMzKxJRqO3U8NU6YitB4aBU4sUjoiHW54+FWhea5mZmZmZWXd8Q+dxqg4kfC5wjKStihSW9BZJd5IdEVvUocxCSSOSRu77xfKK6ZmZmZmZmdVPpY5YfpTrQjp0qiYof25E7AK8C3hfhzLDETEUEUPbP2ffKumZmZmZmVld+IjYOClurXkOcALZ6YZFfQV4TYK6zczMzMzMGqdyRywi1gIXk3XGOpL0vJanrwJur1q3mZmZmZk1hI+IjZPqPmJLgFOmKHOKpJcDfwEeBI5LVLeZmZmZmVmjdN0Ri4g5LY/vB2ZPUf5tZetQgo5tqHoMSJOLmfXGoH7OB3W5UkjVNtYcydZ5qjipPlcDuC2PzkyzUPu/ZWGSONedO5wkzv5vTpPPJo8P4E65qNHR6c6gVlJcI2ZmZmZmZtZYkuZKukrS7fnfbSYoM1/SMkm3SLpZ0utaXjtf0l2SVubT/KnqTN4Rk3R6SwJj0+mp6zEzMzMzswap9zVi7waujojnAVfnz9s9BrwhIl4IHAKcI2nrltdPi4j5+bRyqgpTXSP2VxFxFnBW6rhmZmZmZmY9chhwYP74AuAasltu/VVE3Nby+DeSfgs8DfhDNxV2fURMUkha0vJ8saQzJin/Dkk/yw/jXS1pp27rNjMzMzOzhqn3EbFnRMS9WZpxL/D0yQpL2hvYFLizZfZZeV/nY5I2m6rCKqcmrgMOl7RdwfIrgKGI2AO4FPhwhbrNzMzMzMz+StJCSSMt08K2178vafUE02El63kmcBHwxogYG4HkPcDzgRcDc2k7mjaRKqcmrgeGgVOBKa8Bi4gftjxdDhxboW4zMzMzM2uS0d6OGBkRw2T9k06vv7zTa5Lul/TMiLg372j9tkO5LYHvAO+LiOUtse/NH66T9AVg8VT5Vh2s41zgGElblXzfCcB3J3qhtSd7313LJypiZmZmZmaW0lKeuM/xccC32gtI2hT4BnBhRFzS9toz878CXgOsnqrCSh2xiHgYuBBYVPQ9ko4FhoCPdIg5HBFDETG0/bP3rZKemZmZmZnVRMRoT6eKPgS8QtLtwCvy50gakvTZvMxRwP7A8RMMU/8fklYBq4DtgH+bqsIUoyaeA9wIfGGqgpJeTnYa4wERsS5B3WZmZmZmZpVExAPAggnmjwD/nD/+IvDFDu8/qGydle8jFhFrgYvJTjfsSNKewKeBV0fEhOdcmpmZmZnZgBqN3k4Nk+qGzkvIDsFN5iPAHOCS/DDe0kR1m5mZmZmZNUrXpyZGxJyWx/cDs6co33GUEjMzMzMzG3DV7/U1UFJcI9YzqnzNHah6iMGW4gMht7J1J2q26dQtn1TqtFzy/2CbZlo/mBthis95qs9nzEiz00m179r/zQunLlTAdZ/qOCp6KQecfGKSONZ8yTtikk4HjmybfUlEnJW6LjMzMzMza4jRBEdZBkjyjlje4XKny8zMzMzMnuBTE8dJNViHmZmZmZmZFdR1R0xSSFrS8nyxpDMmKb+/pBslrZd0RLf1mpmZmZlZ88ToaE+npqlyRGwdcLikqYatH/Mr4HjgSxXqNDMzMzMza7wqHbH1wDBwapHCEXF3RNwMNK+7amZmZmZm1UT0dmqYqteInQscI2mrFMkASFooaUTSyH13L08V1szMzMzMrDYqdcQi4mHgQmBRmnQgIoYjYigihrbfed9UYc3MzMzMbDqNRm+nhkkxauI5wAnAUxPEMjMzMzMzG3iVO2IRsRa4mKwzZmZmZmZm9mQx2tupYVLdR2wJMOnoiZJeLGkNcCTwaUm3JKrbzMzMzMysUTbp9o0RMafl8f3A7CnK3wDsWKqOQbzddKrTV5UoTrpA1oE2THcGNSZvf5OJRM2jBPudVLkk2wemkmqULW/LjTHj8eb9al5ErbbArr9djjc6M81SbfJ4ms/5ASefmCTOted9JkkcOC1RnP6JBl7H1UuD2NUxMzMzMzOrtUS/WTxB0ulkpx+2uiQizkpdl5mZmZmZNUQDr+PqpeQdsbzD5U6XmZmZmZlZB12fmigpJC1peb5Y0hmTlN9M0lcl3SHpx5J27rZuMzMzMzNrlhiNnk5NU+UasXXA4ZImHS2xxQnAgxHxXOBjwNkV6jYzMzMzM2usKh2x9cAwcGrB8ocBF+SPLwUWSB5myszMzMxso+D7iI0XEV1NwKPAlsDdwFbAYuCMScqvBnZseX4nsN0E5RYCI/m0sEAeU5YpuDyV49QpF8fxOnccr3PH8Tp3nObn4jhe554Gd6o0fH1EPAxcCCwqUHyio19POpkzIoYjYiifhgvEXVigTBEp4tQpF8fpT5w65eI4/YlTp1wcpz9x6pSL4/QnTp1ycZz+xKlTLinjWI2luI/YOWTXfz11inJrgHkAkjYhO4q2NkH9ZmZmZmZmjVK5IxYRa4GLyTpjk1kKHJc/PgL4QUQ0b3gTMzMzMzOzilIcEQNYAkw1euLngG0l3QG8A3h3orqLnL7Yrzh1ysVx+hOnTrk4Tn/i1CkXx+lPnDrl4jj9iVOnXBynP3HqlEvKOFZj8kEpMzMzMzOz/kp1RMzMzMzMzMwK2iR1QEmnA0e2zb4kIs5KXZeZmZmZmVkT+dREMzMzMzOzPvOpiWZmZmbWd5L2mu4c6kjSXEnbTHce1nuN6ohJmiHpTZK+I+kmST+V9BVJB5aM8xxJn5f0b5LmSPqMpNWSLpG0c4k4z5L0lPyxJL1R0ick/Ut+r7SuSfpgl+/bXtL2+eOnSTpc0gtLvH8TSSdJukLSzXk7f1fSyZJmlYgzM49zpqSXtr32vuJLBJK2lLTLBPP3KBFjj5bHsyS9T9JSSR+UNLtkPpXauCVOiuWq3M6SXj22Hacm6bYu3nOKpO3yx8+VdJ2kP0j6saTdE+X1ihJlU+0vkmyDkr4u6VhJc4q+Z4IYsyX9T0mnSXqKpOPzXD5cMe6z88/D80u+r/Iy5XH6se0UHsksZTtL2krS6yS9Q9Kp+eOtS7w/yb59ijoKf646vL/0/iJ/3xxJR+Tt8lZJh0hK9v2mzDrPy+8v6W/zx/8gabGkV5WMker7TqXtZpK4pb+jSNqrbXoRsFTSnirRIZP0/Hzb/Y6kXSSdn3/OfyJp1xJx5uVter2k97Z+DiR9s+TidapjVYmyz8rz+R3wY+AGSb/N5+2cIh+rn0admijpC8Avge+T3YvsYeB64F3AtyLiEwXjXAd8meym0scCXyC7F9orgWMi4qCCcVYDe0fEY5LOBnYBvgkcBBARbyoY5+Pts4DXAxfmcRYVjHMS2W0BBJwNHA/cArwU+HBEfK5AjC8DfwAuILsJN8COZPeAmxsRryuYy2eB2cBP8mW5NiLekb92Y0QU2ulKOorspuG/BWYBx0fEDV3E+WtZSUuAbcnW+2uAbSPiDQXjVG7jxMtVuZ0l/Qn4I/Bdss/FlRGxoUj9bXEeAcZ2KMr/zgYeAyIitiwY55aIeGH++DvAZyPiG/kXkLMi4qWTBihWx68i4lkFy6baX6TaBn8NLCPbz3w/z+07EfF4kffnMS4G7gE2B/4WuDVfpkOB7SPi9QXjfDMiXpM/Poxsm74G+Hvg3yPi/H4tUx4nybYjaW6nl4CbImLHgnFStfMbgPcD3wN+nc/eEXgF8L8i4sICMZLs26eoo8znKtX+4ijgNOAm4GXAj8h+ZN6d7PNZ6ItwwnV+DrA32TX4VwILyPatBwArIuK0gnEqf99Jsd3kcVJ9RxkFlgPrWmbvm8+LEvvS64CPAHOAD5G1yVeB/wq8PSIWFIxzFfC1vP4TgBcBh0bEA5JWRMSeBeMc3ukl4LyIeFrBOMvI9qGXjv0PljSTbNyFt0fEvkXiWMNERGMm4Oa258vzv5sBt5aIs6Ll8a86vVYgzs9aHv8UmNHy/KYScdYAXwTeQPZP8Tjgd2OPS8RZRfaPbFvgUbJ/9ADbACsLxvj5JK/d1s26IvuHNAx8PV9XZdp4JfDM/PHewP8DDu9iXa1oizkrf6z27arXbZx4uSq3M7Aiz/9E4GrgfuA84ICieeRxPkH2j/kZLfPuKhOjfRsEbui0vAXiLO0wXQb8scttp8r+ItU2uCL/uwXZl6HL8/3FF4BXFt3+Wuq+jyd+lOsql/zxj4Bn54+3o9w+sPIyJd52NgC/AO5qmcaeP14iTqp2/jmw9QTzt6Hgfpl0+/ZUn6tU+4ubgdkt292V+eM9gB9Nwzq/JV+/s4EHW3KbBawus1xtz0t/30mx3eTlU31HOQK4Fviniuu8db9zR9trN5aIs7Lt+bH5+tulZJy/AOfn+6v26ZEScW7v5jVPzZ6Sj5rYY3+RtEtE3Jkfxn4cICLWSSpzaG9U0t+Q/cI9W9JQRIxIei4ws0SceyQdFBE/AO4G5gG/lLRtiRgAuwJnAocAp0XEryW9PyIuKBnnLxHxGPCYpDsj4j6AiHiwRPs8KOlI4GsRMQrZKRJkv8g8WCKXTcceRMR6YKGkfwV+QPYrVlEzI+LePM5PJL0M+LakHXni19QitpL0WrJfSjeLiL/kMaPktpOijSHdcqVo54iIB4HPAJ9RdtrlUcCHJO0YEfMKBnlrfqrJl/PTOj5ZclnGXCrpfOADwDckvZ2sc7kA+FWJOPuR/WN9tG2+yDq/RaXaX6TaBiN/3yPARcBF+a/5R5Edrf1e4UBZ3ZdHRLQ8L51LbpOIuCuP8/v81+9ScRIsU6pt5xfAgoh40nsk3VMiDpCkncXEn6VRnjiaNJVU+/Ykn6uE+wsBf8of/xF4eh7/ZkmFjqrlUq3zyNfv2PY/tkyjlLscJMX3nRTbDST6jhIRl0q6AjhT0huBd3bIbyqt+92Ptr22KcXNkvSUiPhznt8XJd1HdiTzqSXi3Az874hY3f6CpJeXiPNTSZ8iO2o9ts3NI+vwrigRx5pkunuCZSay01Z+BdxG9ivVPvn8p5GdFlY0zgKyX4puBf6B7ND0HWSniR1WIs484IfAdWS/Bj5I9gV4BdkOvezyvSiPtxi4u4v3j/DEr+w7tsx/CgV/nQZ2JjvE/7u8nW/L2+Wr5L92F4zzReCQCeb/M1lnpmicHwG7tM3bguzIzboScdp/pXpGPn974Op+tnHi5arczkxyVAfYqYvtcAawiOw0mt+UfX8e43iyc+R/DzwC/Az4ILBViRjfBV7W4bXrSsRJtb9ItQ0Wzn2SGJ8F5kwwfxfgP0vE2UB2ytQjZF8Ux44Qb0q5Iz6VlynxtvMW4O86vPbWaWjn44A7gf8DvDefzsvnHV8wxs6k2bcn+Vy1vKfS/oLsFPEr8za5HnhvPn8ucMs0rPOz8zxuIDt97jLgdLIfE84rEafy950U201bvErfUdpizc9j/baL957U4XP1XOCcEnFOZYIzP4A9gatKxNkPeFaH14ZKxNkU+BfgCrKzb1bnn7c3k/1413V7e6rv1KhrxAAkiex6it8njrsd8GB0d23MrsDfkJ0atobslJgyvwa3xhLZh+4lEXFsyfc+i+wf2fq2+TsAu0bE90vG25bsVJqkbV0yh78jO93ljrb5s4CjIuI/+pxPkjau03JJOjAirulB3GcCe0bE5aljT7cq+4u6k6So+I9B2WAAu0bEskRpDZyy7axsBLWDgR3IjmasITsNr8zRrLFY075vb1dlfyHpn4AXkP0YdlU+bwbZj2brJn1zD0h6CdmRseXKBmR6LVmn6tIy3w1SfN9Jud205NTVd5QOsbaIiIfb5r8nIv69SmzHscaY7p5g2QnYkrYjCfn8PaYjTsG6ljUpzqC2cd3y6XUc4BWD1sapl6kubVw2Ti/bueltXLf9VxP2O01f585n+nLpUZzC12c5jqemT00bvv4oskENvibpFkkvbnn5/H7HKSHV0OA9jzOobVy3fPoUp9AIjiXUoY1TLxPUo40Lx+lDOze2jeu2/2rQfqex67wE59NZHf5ftSpz/ZrjWKM1bbCO9wIvioh7Je1NdkH3eyPi65TbMFPFKSrV+Z/9iDOobVy3fJLEkbS000tkIzum1Jc27vMyQZ/aOOFyVW7nQW1j6rf/qs1+Z4DXeVHOp8tcHGdg41gNNK0jlmqkuVRxBtGgtnHd8kkl1ciAKaRq4zotE6TLJ1WcFO08qG1ct/1XnfY7g7rOU3E+9VG3I0eDGsdqoGkdsUeUD+cKkP/KeCDZTZRfOA1xiqrbh2+yOIPaxnXLJ1Wc5cBjEXHtk94o/TxRDkVzSdXG/Vwm6F8bp4qTop0HtY3rtv+q035nUNe58+l9Lv2Oc4nj9CWO1UCjRk1UopHmUsWZJP5M4OixOJJ2iwnuL1HHOIPaxr3OZ4L6ulpXvYrTZd21buNUprONu9GEdk617+qi3lrtv5q63+my7mlZ586nPrkU+B/xr5O8PSLizIL1OI4NjEZ1xIqStCwiXtLrOMpuFvkWsmFhlwJXAaeQ3WNjZUQcVrCeWsUpWFej2jhhPo8w8SlFIttRFrqBaKo4RU22XHVr41Rx6tTGTY3TtM/noMbp57bctHXufHq/b0/4f++dE8yeTXbfy20jYo7jVI9jzdK0UxOL6tcIQBeR3cR5GdkH5TSyG/IdFhErS9RTtzhFNK2Nk+QTEVukqCRVnBImW65atXGqODVr46bGadTnc1Dj9Hlbbto6dz49ziXh/70lY48lbQG8DXgT8BVgSaf3OU65ONYsg9oR69fINM+JiN0BJH0W+D3Z3dUfKVlP3eIU0bQ2TpVPU022XE1t47qtq7otV4o4g/r5HNQ4KTRtnTuf+uQyJUlzgXcAxwAXAHtFdzcldxwbCIPaEeuXv4w9iIgNku7qcgdXtzh1MojLVDduY+uWt52NT93WufNpRi5I+ghwODAM7B4R7SNCOk6CONYsg3qN2IqI2LPXcSRtAP449hTYHHiM8udN1ypOwboa1cap8mmqyZarqW1ct3VVt+VKEWdQP5+DGieFpq1z51OffftUJI0C64D1jD+SV/a7juPY4IiIxk/ATOCYlue7TWecQZwGtY3rls8gLpe3nebEcRs3K06d2tj5bJz5ePLkqdo0Y/JuWr1I2lLSeyR9UtIrlXkr8AvgqLFyMcUwrKniDKJBbeO65ZNKnZbL205/litFHLdxs+KkUKdcnE/z8jGz3mjUqYmSvsUTIwAtALYhGwHobVFiBKBUcQbRoLZx3fJJpU7L5W2nOXHcxs2Kk0KdcnE+zcvHzHqjaR2xVfHECEAz6XIEoFRxBtGgtnHd8kmlTsvlbac5cdzGzYqTQp1ycT7Ny8fMeqNRpybSNgIQcFeXO6VUcQbRoLZx3fJJpU7L5W2nOXHcxs2Kk0KdcnE+zcvHzHqgaUfEGje6YNMMahvXLZ9U6rRc3naaE8dt3Kw4KdQpF+fTvHzMrDca1REzMzMzMzMbBE07NdHMzMzMzKzx3BEzMzMzMzPrM3fEzMzMzMzM+swdMTMzMzMzsz5zR8zMzMzMzKzP/j90fs9cLPKgZQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x648 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(16,9))\n",
    "sns.heatmap(corr0,cmap='viridis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "I_0     0.971677\n",
       "I_1     0.974881\n",
       "I_2     0.969608\n",
       "I_3     0.955879\n",
       "I_4     0.969420\n",
       "N_0     0.390693\n",
       "N_1     0.392505\n",
       "N_2     0.392618\n",
       "N_3     0.394198\n",
       "N_4     0.389419\n",
       "R_0     0.963157\n",
       "R_1     0.968155\n",
       "R_2     0.950993\n",
       "R_3     0.951054\n",
       "R_4     0.951443\n",
       "R_5     0.970516\n",
       "R_6     0.951103\n",
       "R_7     0.963075\n",
       "R_8     0.963193\n",
       "R_9     0.968416\n",
       "R_10    0.968274\n",
       "R_11    0.962994\n",
       "R_12    0.963285\n",
       "R_13    0.970870\n",
       "R_14    0.968603\n",
       "R_15    0.970800\n",
       "R_16    0.970886\n",
       "R_17    0.968272\n",
       "R_18    0.970646\n",
       "R_19    0.963149\n",
       "R_20    0.951018\n",
       "R_21    0.968572\n",
       "R_22    0.951056\n",
       "R_23    0.968400\n",
       "R_24    0.963095\n",
       "R_25    0.951220\n",
       "R_26    0.968266\n",
       "R_27    0.970673\n",
       "R_28    0.951441\n",
       "R_29    0.951468\n",
       "dtype: float64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "silh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see there is very low correlation among clusters. Hence, we need not to tranform anything in this dataset. The silhouette score also confirm the same, as there no features have silhouette score below zero."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us artificially generate a dataset that can introduce features with low silhouette score. Here the sigmaStd argument of get_classification_data will help us to generate a dataset with high substitution effect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "#we set the value of sigmaStd to 4 to introduce high substitution effect.\n",
    "X_, y_ = get_classification_data(n_features=40,n_informative=5,n_redundant=30,n_samples=1000,sigmaStd=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Now lets check if we obtained our desired dataset \n",
    "corr0,clstrs,silh = get_onc_clusters(X_.corr())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: ['I_1',\n",
       "  'I_4',\n",
       "  'N_4',\n",
       "  'R_5',\n",
       "  'R_8',\n",
       "  'R_11',\n",
       "  'R_13',\n",
       "  'R_15',\n",
       "  'R_16',\n",
       "  'R_18',\n",
       "  'R_19',\n",
       "  'R_27'],\n",
       " 1: ['I_0',\n",
       "  'I_2',\n",
       "  'I_3',\n",
       "  'N_0',\n",
       "  'N_1',\n",
       "  'N_2',\n",
       "  'N_3',\n",
       "  'R_0',\n",
       "  'R_1',\n",
       "  'R_2',\n",
       "  'R_3',\n",
       "  'R_4',\n",
       "  'R_6',\n",
       "  'R_7',\n",
       "  'R_9',\n",
       "  'R_10',\n",
       "  'R_12',\n",
       "  'R_14',\n",
       "  'R_17',\n",
       "  'R_20',\n",
       "  'R_21',\n",
       "  'R_22',\n",
       "  'R_23',\n",
       "  'R_24',\n",
       "  'R_25',\n",
       "  'R_26',\n",
       "  'R_28',\n",
       "  'R_29']}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clstrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, lets see if there is any features with low silhouette score. If yes then we can correct it with the transformation mentioned above (transformation is appiled automatically)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3 feature/s found with low silhouette score Index(['N_0', 'N_4', 'R_0'], dtype='object'). Returning the transformed dataset\n"
     ]
    }
   ],
   "source": [
    "#this function has built-in detection property that detects the features with low silhouette score and corrects it with transformation\n",
    "clusters = get_feature_clusters(X_,dependence_metric=dep_matrix,distance_metric=None,linkage_method=None,n_clusters=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have got the dataset with some features that has some negative silhouette score. Due to this all of noisy features are placed with the informative and redundent feature clusters. **This is the caveat of the ONC algorithm**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['I_0',\n",
       "  'N_0',\n",
       "  'N_4',\n",
       "  'R_1',\n",
       "  'R_9',\n",
       "  'R_10',\n",
       "  'R_14',\n",
       "  'R_17',\n",
       "  'R_21',\n",
       "  'R_23',\n",
       "  'R_26'],\n",
       " ['I_2',\n",
       "  'I_3',\n",
       "  'N_1',\n",
       "  'N_2',\n",
       "  'N_3',\n",
       "  'R_2',\n",
       "  'R_3',\n",
       "  'R_4',\n",
       "  'R_6',\n",
       "  'R_7',\n",
       "  'R_12',\n",
       "  'R_20',\n",
       "  'R_22',\n",
       "  'R_24',\n",
       "  'R_25',\n",
       "  'R_28',\n",
       "  'R_29'],\n",
       " ['I_1',\n",
       "  'I_4',\n",
       "  'R_0',\n",
       "  'R_5',\n",
       "  'R_8',\n",
       "  'R_11',\n",
       "  'R_13',\n",
       "  'R_15',\n",
       "  'R_16',\n",
       "  'R_18',\n",
       "  'R_19',\n",
       "  'R_27']]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clusters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the composition after transformation has changed and now we have 3 clusters instead of 2. Though this is not perfect but it has done a must better job in clustering than the normal ONC algorithm. Also the get_feature_clusters function can detect the problem of low degree of freedom of the regression model used for generating the residual $\\hat \\varepsilon _{i}$ for replacing the orginal feature $X_{i}$ as mentioned above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Hierarchical Clustering "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There is no feature/s found with low silhouette score. All features belongs to its respective clusters\n"
     ]
    }
   ],
   "source": [
    "dist_matrix = 'angular' # angular distance matric\n",
    "linkage = 'single' # linkage method for hierarchical clustering\n",
    "clusters_ = get_feature_clusters(X,dependence_metric=dep_matrix,distance_metric=dist_matrix,linkage_method=linkage,n_clusters=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['I_0',\n",
       "  'I_2',\n",
       "  'R_1',\n",
       "  'R_7',\n",
       "  'R_9',\n",
       "  'R_10',\n",
       "  'R_12',\n",
       "  'R_14',\n",
       "  'R_17',\n",
       "  'R_21',\n",
       "  'R_23',\n",
       "  'R_24',\n",
       "  'R_26'],\n",
       " ['I_1',\n",
       "  'I_4',\n",
       "  'R_0',\n",
       "  'R_5',\n",
       "  'R_8',\n",
       "  'R_11',\n",
       "  'R_13',\n",
       "  'R_15',\n",
       "  'R_16',\n",
       "  'R_18',\n",
       "  'R_19',\n",
       "  'R_27'],\n",
       " ['I_3',\n",
       "  'N_0',\n",
       "  'N_3',\n",
       "  'R_2',\n",
       "  'R_3',\n",
       "  'R_4',\n",
       "  'R_6',\n",
       "  'R_20',\n",
       "  'R_22',\n",
       "  'R_25',\n",
       "  'R_28',\n",
       "  'R_29'],\n",
       " ['N_2'],\n",
       " ['N_1'],\n",
       " ['N_4']]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clusters_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using Heirarchical Clustering we get 6 clusters and 3 of them are with only single element and are non-informative features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the end I would like say that the user can use different depedence matric including both correlation based and information theory based metric (like Information Variation which have the advantage of recognizing redundant features that are the result of nonlinear combinations of informative features). Also the user can use different linkage methods for hierarchical clustering or define the number of clusters."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
